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ABSTRACT 


The  Defense  Logistics  Agency  Disposition  Services  (DLA  DS)  is  facing  substantial 
potential  budget  cuts,  which  will  require  the  organization  to  increase  efficiency  and  cut 
costs.  DLA  DS  is  assessing  the  feasibility  to  improve  the  item  disposal  process  with  an 
object/shape  recognition  technology  as  a  means  to  increase  efficiency.  The  technology 
may  potentially  reduce  the  time  required  to  identify  items  in  order  to  route  for  appropriate 
disposal.  With  the  advancement  of  new  technology  and  the  availability  of  databases,  it 
may  now  be  possible  to  identify  an  object  using  recognition  technology.  Given  the 
potential  of  these  technologies  to  reduce  identification  time  and  errors,  research  to  assess 
the  viability  of  existing  prototypes  is  justified.  This  applied  project  identified  and 
assessed  object/shape  recognition  technologies  as  a  means  to  support  efficiency  while 
taking  into  account  wider  government  policy  and  objectives  in  relation  to  technology 
adoption.  This  project  evaluated  the  feasibility  of  implementation  of  these  technologies 
that  includes  an  analysis  of  DLA  DS’s  items  disposal  process,  a  technology  readiness 
assessment  of  the  technology,  and  a  cost-benefit  analysis  to  assess  the  financial  payoffs 
of  an  investment  in  the  technology.  The  objective  was  to  understand  the  items  disposal 
process  and  assess  object/shape  recognition  technology  to  address  the  problem  that 
unlabeled  items  poses  to  DLA  DS  at  Camp  Pendleton,  CA. 
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EXECUTIVE  SUMMARY 


The  Defense  Logistics  Agency  Disposition  Services  (DLA  DS)  is  facing  substantial 
potential  budget  cuts,  which  will  require  the  organization  to  increase  efficiency  and  cut 
costs.  DLA  DS  conducted  a  Lean  Six  Sigma  study  in  2008  that  led  to  streamlining  the 
item  disposal  process  and  saving  time  equivalent  to  40%  of  a  full  time  employee.  The 
Lean  Six  Sigma  study  concluded  with  a  recommendation  to  identify  and  study  the 
potential  of  technologies  to  support  further  efficiency  gains  (DLA  DS,  2008). 

This  master’s  thesis  was  motivated  by  that  recommendation.  In  the  past  decade, 
several  companies  have  developed  object/shape  recognition  systems  to  meet  specific 
commercial  requirements,  with  some  success  (Rajpurohit  et  ah,  2013).  The  objective  of 
this  study  was  to  identify  and  assess  the  viability  of  object  recognition  technologies  for 
use  by  DLA  DS. 

This  study  is  a  mixed-method  design  including  analysis  of  qualitative  and 
quantitative  data.  Specifically,  this  study  includes  an  analysis  of  DLA  DS’s  items 
disposal  process,  a  technology  readiness  assessment  (TRA)  of  object/shape  recognition 
technology,  and  a  cost-benefit  analysis  (CBA)  to  assess  the  financial  payoffs  of  an 
investment  in  the  technology.  The  TRAs  and  CBA  are  consistent  with  the  Circular  A  94, 
DOD  Directive  5000.2  Interim  and  DOD  TRA  guidance  (OMB,  1992;  DOD,  2013a; 
ASD[R&E],  2011).  The  study  assessed  a  four-month  period  of  data  and  utilized  a  market 
comparable  approach  to  calculate  the  benefits  and  costs  and  return  on  investment.  As  a 
public  service  sector  organization,  DLA  DS  does  not  generate  revenues.  The  study  used 
the  market  comparable  labor  to  estimate  revenues. 

This  thesis’  objective  was  to  understand  the  items  disposal  process  and  assess 
object/shape  recognition  technology  to  address  the  problem  that  unlabeled  items  poses  to 
DLA  DS  at  Camp  Pendleton,  CA.  However,  the  findings  generated  by  the  analysis  of 
DLA  DS’s  items  disposal  processing  show  three  other  areas  of  concern  that  contributed 
to  days’  worth  of  backlogged  items.  The  data  analysis  suggests  that  the  problem  is  not 
exactly  the  unlabeled  items.  Indeed,  employees  spend  excess  time  conducting  research  on 
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items  that  are  not  labeled  or  are  missing  DD  1348  forms.  However,  unlabeled  items  are 
fewer  than  1%  of  accepted  items,  and  handling  them  does  not  solely  ’create  a  backlog. 
The  following  three  other  areas  of  concern  also  contributed  to  days’  worth  of  backlog. 
First,  improperly  filled  DD  1348  provides  for  additional  time  spent  conducting  research 
in  order  to  fill  the  DD  1348  form  for  further  processing.  Second,  the  receipt  control 
numbers  (RCNs)  are  not  limited  to  the  amount  of  items  that  can  arrive  in  a  pallet  or  tri¬ 
box.  A  RCN  can  have  one  or  50  items;  therefore  on  average  the  facility  accepts  425  items 
per  day.  The  DLA  DS  at  Camp  Pendleton’s  capacity  to  process  items  is  limited  to  300 
items  with  six-employee  crew.  Therefore,  third,  DLA  DS  at  Camp  Pendleton  accepts 
more  items  than  what  it  can  process.  The  facility  processes  an  average  of  300  items  per 
day;  however,  the  excess  items  that  were  not  processed  contributed  to  have  an  average  of 
47  RCN  equaling  to  about  2,259  items  or  7.5  days’  worth  of  backlogged  items  stored  for 
later  processing. 

Furthermore,  the  thesis  examined  the  alternative  of  addressing  the  unlabeled  items 
issues  with  the  use  of  new  object/shape  recognition  technology.  The  DLA  DS 
technological  requirement  includes  item  recognition  and  disposal  information  feedback, 
and  the  ability  to  take  pictures  to  use  on  DLA  email  system  to  communicate  and  upload 
the  picture  to  liquidations  website.  The  TRA  for  three  commercial-off-the-shelf 
technologies  shows  that  they  do  not  meet  the  maturity  standard  of  TRL  7  required  for 
DOD  projects.  However,  Imaginestics’  technology  is  the  most  viable  and  mature  system, 
ranking  up  to  TRL  5.  Imaginestics’  technology  can  be  implemented  in  less  considerable 
time  and  cost  in  reference  to  the  other  two  companies. 

The  results  of  the  CBA  shows  a  total  quantified  net  economic  benefit  occurring 
from  the  technology’s  contributions  to  be  an  estimated  $65,000  corresponding  to  an 
internal  rate  of  return  of  15%  over  a  10-year  time  span  and  profitability  index  (PI)  1.43, 
indicating  that  for  every  dollar  invested  $1.43  in  benefits  is  accrued  with  the  payback 
period  of  almost  seven  years.  The  new  technology  will  save  money  in  labor  costs  for  the 
unlabeled  items;  however,  it  only  address  less  than  1%  of  items  and  not  the  7.5  days’ 
worth  of  backlogged  items  for  the  DLA  DS  facility  located  in  Camp  Pendleton. 
Therefore,  the  results  from  the  queuing  theory  suggests  that  with  nine  employees  the 
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DLA  DS  at  Camp  Pendleton  will  keep  up  with  demand  by  processing  all  received  items 
and  will  progressively  reduce  the  7.5  days’  worth  of  backlogged  items. 

Areas  of  uncertainty  remain  for  future  study.  This  study  was  not  able  to  capture 
time  spent  in  every  step  of  the  item  disposal  process,  or  observe  whether  the  backlog  is  a 
result  of  specific  events  such  as  retrograding  units  from  deployments  or  the  time  of  the 
year.  The  study  did  not  assess  the  potential  added  value  lost  of  scrapped  items  that  might 
have  been  sold.  For  example  if  the  technology  increased  the  items  sent  for  liquidation 
there  would  be  an  increase  in  revenues  to  DLA.  Additionally,  data  was  collected  at  one 
site,  the  Camp  Pendleton  DLA  DS  facility,  such  that  the  results  may  not  be  indicative  of 
all  DLA  DS. 

One  of  the  main  conclusions  resulting  from  this  thesis  effort  is  that  researching 
items  is  not  the  only  area  of  concern  with  regard  to  process  inefficiency.  Rather,  this 
thesis  identified  three  other  areas  of  concern  suggested  by  the  data  analysis  and 
observation.  Therefore,  further  recommendations  were  provided  to  take  care  of  the  other 
three  problems. 

The  TRA  and  CBA  show  that  the  implementation  of  object/shape  recognition  is 
viable,  however  the  technology  will  address  delays  associated  with  the  less  than  1%  of 
the  backlogged  items  at  Camp  Pendleton.  This  study  recommends  DLA  DS  take  steps  to 
address  additional  identified  concerns.  This  study  recommends  that  DLA  DS: 

1.  Making  mandatory  to  fill  the  DD  1348  on  DLA  DS  website,  and 

2.  Limiting  the  amount  of  items  on  pallets  to  match  the  facility’s  processing 
capacity,  and 

3.  Increasing  the  Camp  Pendleton  facility  to  nine  employees,  or 

4.  Adding  a  night  shift  such  that  the  shift  only  process  items  without 
accepting  or  conducting  mundane  tasks,  or 

5.  Once  a  month  stop  accepting  items  for  disposal  process  for  a  full  week  in 
order  to  focus  the  effort  to  dispose  the  backlog  items,  or 
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6. 


Fill  the  gap  with  technology  by  implement  full  automation  with  optical 
sorting  and  data  mining  that  included  sensors,  laser,  object/shape 
recognition  technology  on  conveyor  belt  system  and  data  analytics  to 
improve  materiel’s  opportunity  to  be  sold  as  commodity,  or  liquidations, 
and  improve  operational  efficiency;  which  would  be  consistent  with  DLA 
Director’s  “Big  Idea”  and  DLA  DS’s  strategic  technological  goal  to 
accomplish. 

The  technological  innovation  disrupts  status  quo  by  ensuring  a  plan  of  increased 
capacity  for  the  long  run  and  perhaps  reduces  some  of  the  over  $60  million  of  labor  cost 
that  account  for  about  35%  of  DLA’s  budget. 
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I.  INTRODUCTION 


A.  OVERVIEW 

The  budget  cuts  the  Department  of  Defense  (DOD)  proposes  to  implement  are 
creating  a  spending  habit  shift  of  the  Defense  Logistics  Agency  Disposition  Services 
(DLA  DS).  The  budget  cuts  will  require  the  organization  to  increase  efficiency  and  cut 
costs.  DLA  DS  conducted  a  Lean  Six  Sigma  study  in  2008  that  led  to  streamlining  the 
item  disposal  process  and  saving  time  equivalent  to  40%  of  a  full  time  employee.  The 
Lean  Six  Sigma  study  concluded  with  a  recommendation  to  identify  and  study  the 
potential  of  technologies  to  support  further  efficiency  gains  (DLA  DS,  2008). 

This  master’s  thesis  was  motivated  by  that  recommendation.  In  the  past  decade, 
several  companies  have  developed  object/shape  recognition  systems  to  meet  specific 
commercial  requirements,  with  some  success  (Rajpurohit  et  ah,  2013).  The  objective  of 
this  study  was  to  identify  and  assess  the  viability  of  object  recognition  technologies  for 
use  by  DLA  DS  to  increase  efficiency  and  reduce  costs.  This  chapter  describes  the 
problem  facing  DLA  DS,  which  motivated  this  study;  details  the  research  questions  that 
guided  the  study;  provides  a  description  of  DLA  DS  technology  requirements;  and 
outlines  the  organization  of  this  report. 

B.  THE  PROBLEM  STATEMENT 

According  to  the  Government  Accountability  Office  (GAO,  2010),  the  Defense 
Logistic  Agency  (DLA)  Dispositions  Services  (DS)  identified  $1  billion  worth  of 
materiel  for  reutilization  or  disposal  between  fiscal  years  (FY)  2006  and  2008.  However, 
identifying  some  materiel  to  allow  reutilization  or  disposal  has  been  difficult  to 
efficiently  accomplish.  Time  spent  researching  to  identify  material  could  be  a  key  factor 
driving  backlogs  at  DLA  DS.  Additionally,  the  research  process  may  result  in 
inefficiency  and  decrease  goal  attainment.  In  2008,  DLA  DS  initiated  steps  to  improve 
the  identification  process  by  improving  process  flow,  but  further  opportunities  for 
improvement  might  be  accomplished  with  technology  (DLA  DS,  2008). 
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In  general,  successful  operations  enhancement  with  technology  saves  money  by 
providing  faster  service,  reducing  backlogs  and  reducing  the  cost  of  storing  items. 
However,  implementing  technology  has  associated  costs,  including  the  cost  of  acquiring 
a  technology  that  meets  DLA  requirements  and  also  the  cost  associated  with  developing  a 
technology  for  the  specific  needs  of  DLA  DS.  Development  of  immature  technology 
increases  the  investment  cost  thereby  increasing  the  payback  period  and  reduces  the 
willingness  to  invest.  An  effective  evaluation  of  the  technology’s  maturity  is  required  to 
support  informed  decisions.  While  the  DLA  DS  item  disposal  process  flow  has  been 
improved,  additional  efficiency  improvements  are  desired  (DLA  DS,  2014a).  This 
applied  research  assesses  the  potential  of  object/shape  recognition  technology  to  improve 
the  DLA  DS’s  item  disposal  process  by  facilitating  item  identification.  The  aim  is  to 
evaluate  the  current  state  of  object/shape  recognition  technology  and  assess  the  feasibility 
of  implementing  it  at  DLA  DS. 

C.  RESEARCH  QUESTIONS,  SCOPE  AND  LIMITATIONS  OF  THE  STUDY 

This  study  addresses  the  questions  that  follow. 

1 .  Is  object/shape  recognition  technology  a  viable  tool  to  improve  DLA  DS 
operations? 

2.  What  are  the  technology  needs  for  DLA  DS? 

3.  What  technologies  are  available  and  are  they  mature  enough  for 
implementation? 

4.  How  much  will  the  technology  cost  and  will  it  generate  any  tangible 
savings? 

5.  What  is  the  payback  period? 

1.  Scope  and  Limitations  of  the  Study 

This  study  is  a  mixed-method  design  including  analysis  of  qualitative  and 
quantitative  data.  Specifically  this  study  includes  an  analysis  of  DLA  DS’  items  disposal 
process,  a  technology  readiness  assessment  (TRA)  of  object/shape  recognition 
technology,  and  a  cost  benefit  analysis  (CBA)  to  assess  the  financial  payoffs  of  an 
investment  in  the  technology.  The  TRAs  and  CBA  are  consistent  with  the  Circular  A-94, 
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DOD  Directive  5000.2  Interim  and  DOD  TRA  guidance  (OMB,  1992;  DOD,  2013a; 
ASD[R&E],  2011).  The  study  assessed  a  four-month  period  of  data  and  utilized  a  market 
comparable  approach  to  calculate  the  benefits  and  costs,  and  return  on  investment.  As  a 
public  service  sector  organization,  DLA  DS  does  not  generate  revenues.  The  study  used 
the  market  comparable  labor  to  estimate  revenues. 

D.  DLA  DISPOSITION  SERVICES  TECHNOLOGY  REQUIREMENTS 

In  recent  years,  significant  progress  has  been  made  in  the  development  of 
object/shape  recognition  technologies  and  their  use  in  several  real-time  vision  systems  for 
detecting  specific  classes  of  objects  within  complex  environments.  This  study  assess  the 
potential  of  several  companies’  technologies  to  meet  the  four  technological  needs  that  the 
DLA  DS  requires: 

1 .  Identify  government  property  via  object/shape  recognition  or  barcodes 
with  national  stock  numbers  (NSNs). 

2.  Provide  disposition  instructions. 

3.  Meet  cyber  security  standards  and  information  assurance  requirement. 

4.  Take  pictures  to  use  on  the  DLA  email  system  to  communicate  or  upload 
to  liquidation  website. 

E.  ORGANIZATION  OF  STUDY 

The  remainder  of  this  report  is  organized  as  follows.  Chapter  II  reviews  of 
relevant  academic  research  including,  technology  commercialization,  technology 
transition  and  technology  readiness  assessment  research  as  well  as  literature  from  the 
DOD  to  provide  a  background  and  framework  for  this  study.  Chapter  III  describes  the 
method  used  to  investigate  the  research  questions.  Chapter  IV  describes  the  results  of  the 
analysis  of  Disposition  Services  at  Camp  Pendleton’s  items  disposal  process  and  the  TRA 
of  three  companies’  prototypes.  Chapter  V  describes  the  CBA  for  object/shape 
recognition  available  technologies  and  discusses  the  results.  Finally,  chapter  VI 
summarizes  the  key  findings  of  the  study,  details  recommendations  for  DLA  DS 
supported  by  the  findings,  and  makes  recommendations  for  future  study. 
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II.  LITERATURE  REVIEW  AND  BACKGROUND 


A.  OVERVIEW 

This  chapter  reviews  academic  literature  and  U.S.  government  policies  in  order  to 
provide  a  description  of  the  known  understanding  and  developments  in  this  area.  The 
decision  to  adopt  technology,  particularly  by  government  departments,  is  not  an  easy 
process  and  is  not  simply  technology  assessment  in  isolation.  Many  factors  and  processes 
must  be  taken  into  account.  The  objective  of  the  chapter  is  to  provide  an  overview  of 
concepts  and  terminology  related  to  government  procurement  of  commercial  items.  This 
chapter  reviews  (1)  the  academic  literature  related  to  technology  commercialization  (TC) 
including  challenges  of  developing  and  commercializing  technology,  (2)  studies 
describing  the  generally  accepted  approach  for  cost-benefit  analysis  (CBA),  and,  (3) 
policies,  procedures,  and  guidance  of  the  United  States’  statute  and  law  related  to  the 
acquisition  of  commercial-off-the-shelf  (COTS)  products,  technology  transition,  and 
technology  readiness  assessment.  This  review  provides  a  background  and  suggests  a 
method  to  identify,  evaluate  and  develop  a  solution  to  the  DLA  DS’s  property 
identification  problem. 

B.  UNDERSTANDING  THE  TECHNOLOGY  COMMERCIALIZATION 

PROCESS 

New  technologies  have  great  potential  but  their  commercialization  has  often 
proven  challenging  (Zahra  &  Nielsen,  2002).  Technology  Commercialization  (TC) 
is  of  interest  to  academics  and  industry,  and  it  has  thus  received  a  great  deal  of  attention 
in  the  literature.  Research  has  focused  on  identifying  the  processes  to  successfully 
manage  commercialization;  the  development  and  launch  of  new  products  (Jolly,  1997, 
p.  20).  TC  seeks  to  identify  the  process  through  which  innovative  ideas  are  transformed 
into  marketable  products  and  to  suggest  means  of  overcoming  challenges  to 
commercialization. 
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1.  What  Is  Technology  Commercialization 

TC  is  defined  as  the  “process  of  acquiring  ideas,  augmenting  them  with 
complementary  knowledge,  developing  and  manufacturing  saleable  goods,  and  selling 
the  goods  in  the  market”  (Zahra  &  Nielsen,  2002).  Additionally,  Jolly  (1997)  explained 
TC  as  “taking  a  design  through  development  and  then  manufacture  and  market  it”  (p.  25). 
The  literature  on  TC  suggests  that  successful  technology  commercialization  processes  act 
as  a  technology  accelerator  (Markman,  Siegel,  &  Wright,  2008).  TC  involves  a  series  of 
incremental  steps  to  move  a  technology  along  the  technology-readiness  scale.  As 
explained  by  Jolly  (1997),  bringing  new  technologies  to  market  requires  five  activities: 
imagining,  incubating,  demonstrating,  promoting,  and  adopting.  Mobilizing  stakeholders, 
resources,  and  constituents,  and  delivering  the  product  bridges  the  TC  activities.  The 
steps  of  the  TC  process  should  provide  a  means  to  assess  whether  the  technology  will 
work  in  a  real  application  scenario  as  well  as  to  evaluate  the  technology’s  ease  of  use  and 
determine  how  useful  it  will  be  and  whether  it  is  likely  to  be  sustainable  in  the  market 
(Jolly,  1997).  If  successful,  TC  will  eventually  introduce  new  technology  to  the 
marketplace  and  increase  a  firm’s  competitive  advantage  (Zahra  &  Nielsen,  2002). 
However,  TC  has  proven  challenging  (Jolly,  1997;  Zahra  &  Nielsen,  2002). 

2.  Technology  Commercialization  Challenges 

TC  requires  strategic  decision-making  in  an  uncertain  environment  and  often 
outlaying  upfront  a  large  amount  of  capital.  This  creates  challenges  for  stakeholders, 
which  often  may  prove  too  great  for  to  overcome.  One  of  the  fundamental  reasons  TC 
fails  is  due  to  having  limited  knowledge  of  what  the  customer  wants  or  needs  and  the 
technology’s  acceptance  rate  in  the  marketplace.  According  to  Jolly  (1997),  failure  is  due 
to  the  demand  never  materializing  or  perception  of  poor  product  quality.  Poor  quality 
results  in  a  lack  of  stakeholder  interest  and  support,  because  the  innovation  doesn’t  live 
up  to  the  promised  capacity  (Jolly,  1997).  Investors  may  perceive  that  the  technology  is 
too  risky  or  unlikely  to  result  in  a  worthwhile  benefit,  and  a  lack  of  funding  may  prevent 
research  and  development  (R&D)  and  the  commercialization  process  (Jolly,  1997).  For 
example,  in  the  DOD  the  process  of  budgeting  for  innovation  has  many  constraints  that 
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often  make  it  difficult  to  continue  to  fund  research  and  development  (R&D)  along  the 
complete  pipeline  of  development,  consequently  stalling  projects  for  months  (DOD, 
2003).  This  hiatus  is  known  as  the  “valley  of  death.”  However,  according  Jolly  (Jolly, 
1997),  it  is  important  to  look  at  the  big  picture  and  figure  out  the  links  of  failure  or 
success  in  order  to  prevent  further  failure  to  continue  to  maximize  success. 

3.  What  Makes  Technology  Commercialization  Work? 

Assuming  genuine  innovation  and  strong  market  research,  consistent 
communication  across  stakeholders  and  investor  funding  along  with  controlling  other 
basic  traits  can  fundamentally  strengthen  the  TC  process.  According  Jolly  (1997), 
“successful  innovators  tend  to  exchange  more  information  and  establish  more  efficient 
external  communication  links  than  their  less  successful  counterparts”  (p.  229).  By  taking 
this  approach  network  uncertainty  is  naturally  reduced  through  planning  and  increased 
communication  among  stakeholders.  Good  infonnation  is  another  important  factor.  In 
order  to  gather  information  during  the  TC  process  other  scientists  are  considered  the  first 
customers  of  new  technology  (Jolly,  1997).  They  provide  useful  feedback  to  ensure  that 
technological  adjustments  meet  marketplace  needs.  Close  management  of  the  project 
during  the  early  stages  of  product  design  refines  the  innovation  process  by  monitoring 
and  measuring  resources  used  from  R&D  to  marketing  (Jolly,  1997).  Additionally, 
incorporating  market  needs  with  the  unique  selling  points  to  communicate  how  the  new 
product  is  different  and  offers  new  functions  increases  acceptance  (Jolly,  1997).  Investor 
attraction  requires  a  mix  of  the  scope  of  the  innovation  in  addition  to  the  business  and 
economical  benefit  rather  than  explaining  the  technological  benefit  alone  (Kaarela,  2013). 
These  actions  support  the  successful  commercialization  of  technology,  thus  boosting  the 
confidence  of  stakeholders  and  potential  investors  (Figure  1). 
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the  Process  of  Technology  Commercialization 
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Figure  1.  Technology  Commercialization  Process  (from  Jolly,  1997) 


The  TC  process  posed  by  Jolly  (1997)  illustrates  a  methodical  approach  to 
commercializing  technology,  as  shown  in  Figure  1.  According  to  Jolly  (1997), 
stakeholders’  interest,  recognition,  and  moving  resources  are  key  principles  of 
innovation.  Stakeholder  interest  includes  nine  interconnected  stages.  Five  subprocesses 
build  value  into  the  new  technology  by  “imagining  a  techno-market  insight;  incubating 
the  technology  to  define  its  commercializability;  demonstrating  it  contextually  in 
products  and/or  processes;  promoting  the  latter’s  adoption;  and  sustaining 
commercialization  use”  (Jolly,  1997,  p.  46).  Four  additional  stages  bridge  the 
subprocesses  to  satisfy  and  mobilize  stakeholders  and  resources  along  the  stages.  The 
following  five  subprocesses  are  described: 

1.  Imagining:  idea  stage,  linked  together  the  incubating  stage  with 
“mobilizing  interest  and  endorsement”  (Jolly,  1997,  p.  48). 

2.  Incubating:  the  “defining  moment”  where  the  resources  are  allocated 
(Jolly,  1997,  p.  54);  the  demonstration  stage  is  linked  with  the  mobilizing 
resources  stage. 

3.  Demonstration:  product  development  links  together  the  promoting  stage 
with  mobilizing  market  stage  (Jolly,  1997,  p.  60) 
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4. 


Promoting:  “persuading  people  to  adopt”  (Jolly,  1997,  p.  65)  links 
together  the  sustaining  stage  with  mobilizing  assets  for  delivery. 

5.  Sustaining:  “long  presence  in  the  market  that  generates  fair  shares  on  the 
long  run”  (Jolly,  1997,  p.  69). 

In  summary,  without  recognition  that  all  stages  are  equally  important  in  the 
process — meaning  that,  without  stakeholders’  interest,  appropriate  action  and  moving 
valuable  resources  in  between  every  stage  the  commercialization — technology  often  fails. 

This  section  discussed  Jolly’s  (1977)  model  as  a  tool  in  the  TC.  Communication 
and  information  gathering  reduces  uncertainty  and  gains  the  interest  of  the  investors,  who 
then  provide  project  funding.  Investors  care  about  the  bottom  line  and  about  showing  the 
financial  benefit  that  will  have  to  be  adopted  in  order  to  illustrate  the  financial  cost  and 
benefit  of  the  project.  Additionally,  knowing  enough  of  what  the  marketplace  wants  and 
needs  is  a  function  of  successful  TC.  Finally,  incorporating  Jolly’s  (1977)  model  provides 
guidance  to  facilitate  the  management  of  product  development.  TC  is  successful  when 
challenges  are  overcome  by  including  shareholders  and  investors  in  the  communication 
process  during  every  stage  of  commercialization. 

C.  COST-BENEFIT  ANALYSIS 

A  cost-benefit  analysis  (CBA)  is  a  tool  that  managers  use  to  assess  a  project’s  net 
benefit  in  monetary  tenns  (Boardman  et  ah,  201 1)  During  the  system’s  engineering  phase 
of  the  development  process  the  DOD  Directive  (DODD)  5000.2  Interim,  claims  a  CBA 
needs  to  be  conducted,  which  resolves  the  “...supportability  analysis,  provides  insight 
into  supportability  drivers  and  includes  the  impact  of  resources  on  readiness  supported  by 
engineering  analyses  required  for  product  support”  (DOD,  2013a,  p.  215).  In  other  words, 
all  options  must  be  considered  and  alternatives  must  be  analyzed  to  showcase  which  one 
provides  a  greater  impact  and  solution  to  meet  user  needs. 

In  the  United  States  government,  the  guidance  to  conduct  a  CBA  for  any  federal 
program  is  directed  by  the  Circular  A-94  Guidelines  and  Discount  Rates  for  Benefit-Cost 
Analysis  of  Federal  Programs  (OMB,  1992).  The  Circular  A-94  (OMB,  1992)  provides 
instructional  steps  on  how  to  plan  and  pursue  a  cost  effectiveness  analysis  (CEA)  in  order 
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to  depict  the  risk  associated  with  implementing  a  project  and  lists  the  better  options  that 
out-weigh  benefit  over  risk. 


D.  TECHNOLOGY  TRANSITION 

The  DOD  relies  on  having  technologically  innovative  weapons  systems  in  order 
to  have  an  upper  hand  over  any  adversary.  The  technology  transition  (TT)  process  is  a 
prominently  accepted  method  by  both  private  and  public  sectors  to  create  effective 
technology  systems.  Congress,  the  DOD,  and  universities  have  established  a  number  of 
“technology  transition”  programs  (DOD,  2003;  GAO,  2005).  There  are  many  variations 
of  TT  since  technology  transition  has  been  practiced  from  the  early  1800s  (Geels,  2001). 
TT  by  itself  is  not  the  answer  to  successful  product  launch;  however,  it  paves  the  way  to 
smoothly  and  efficiently  transition  along  the  evolutionary  acquisition  system  (DOD, 
2003).  The  Defense  Advanced  Research  Projects  Agency  (DARPA)  and  the  U.S.  Air 
Force  (USAF)  are  examples  of  organizations  that  have  fully  embraced  TT  processes  to 
innovate  highly  technological  weapons. 

1.  What  Is  Technology  Transition? 

Technology  transition  occurs  when  an  innovative  concept  is  developed  into  a 
technological  tool  the  R&D  phase  then  transferred  to  the  marketplace  for  use  (Markman 
et  al.,  2008).  The  USAF  (2010)  defines  technology  transition  in  a  more  detailed  “in- 
house”  process  that  includes  the  following  activities:  “establishing  a  team,  formulating 
the  plan,  developing  information,  coordinating  and  updating  the  infonnation,  obtaining 
commitment  and  approval  at  the  proper  stages,  and  executing  transition.”  In  other  words, 
the  DARPA  (2010)  refers  to  technology  transition  as  getting  technology  from  research 
and  into  use. 

2.  What  Are  Technology  Transition  Challenges? 

Technology  transfers  is  a  process  that  is  prone  to  failure.  One  reason  for  TT’s 
failure  is  lack  of  oversight  and  supervision.  According  to  the  GAO  (2005),  “failure  to 
track  even  the  most  basic  information,  such  as  the  number  of  projects  completed,  could 
result  in  a  lack  of  ability  to  manage  the  program  properly  and  poor  stewardship  of 
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taxpayer  money”  (p.  25).  This  oversight  contributes  to  instability  of  project  progress  to 
success  and  project  failure.  The  GAO  (2005)  conducted  a  study  on  TT  programs.  The 
study  centered  on  the  management  process,  oversight  and  assessment  practices  for  TT 
programs.  The  research  highlights  many  of  the  ongoing  barriers: 

At  this  time,  however,  the  transition  programs  have  limited  measures  to 
gauge  individual  project  success  and  program  impact  or  return  on 
investment  in  the  long  term.  At  best,  they  are  collecting  after  action 
reports  that  describe  the  results  of  transition  projects,  and  occasionally 
identify  some  cost  savings,  but  not  in  a  consistent  manner.  In  addition, 
there  are  inconsistencies  in  how  the  reports  are  being  prepared,  reviewed, 
and  used.  The  Quick  Reaction  Fund  program  manager,  in  fact,  had  trouble 
just  getting  projects  to  submit  after  action  reports.  (GAO,  2005,  p.  22) 

3.  What  Makes  Technology  Transition  Work? 

Consistent  supervision  can  fundamentally  impact  a  smooth  TT  process.  The  GAO 
(2005)  research  suggested  the  following: 

Selecting  promising  projects  for  funding  is  not  enough  to  ensure 
successful  transition.  Program  managers  must  also  actively  oversee 
implementation  to  make  sure  that  project  goals  are  being  met  and  the 
program  is  working  as  intended  and  to  identify  potential  barriers  to 
transition.  They  must  also  sustain  commitment  from  acquirers.  Moreover, 
the  transition  program  as  a  whole  must  have  good  visibility  over  progress 
and  be  positioned  to  shift  attention  and  resources  to  problems  as  they 
arise,  (p.  18) 

One  of  the  best  approach  that  supports  TT  success  is  to  establish  a  database  that 
facilitates  the  collection  of  pertinent  information  (GAO,  2005).  The  TT  process  posed  by 
the  USAF  (2010)  illustrates  a  methodical  approach  to  technology  transition  as  shown  in 
Figure  2.  The  model  highlights  the  linear  process  from  accepting  or  approving  a  concept 
up  to  transitioning  to  a  prototype. 
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Figure  2.  Technology  Transition  Process  Flow  Diagram  (from  USAF,  2010) 


The  USAF  Technology  Transition  model  in  Figure  2  illustrates  phases  and  stage- 
gates  that  are  significant  to  the  smooth  process  of  TT  and  is  applicable  to  this  research. 
The  figure  illustrates  a  method  to  measure  and  approve  technology  before  moving  to  the 
next  stage.  Although  this  model  is  depicted  in  a  linear  fashion  the  innovation  process  is 
often  a  circular  model  with  many  cycles  of  proof  of  concept  and  development  before  a 
prototype  is  ready.  The  TT  process  published  by  the  USAF  (2010)  illustrates  a 
managerial  approach  to  approve  a  concept  and  transition  technology  from  lab  to  a 
weapon  system.  The  USAF  adopted  the  stage-gate  process  due  to  its  “industry-proven 
mechanism  for  effectively  managing,  directing,  [and]  otherwise  controlling  new  product 
development  efforts”  (USAF,  2010,  p.  8).  The  explanation  of  the  activities  and  stage- 
gates  is  as  follows: 

1 .  Establish  a  team:  Establish  a  team  of  stakeholders,  managers  and 
gatekeepers  in  order  to  make  appropriate  strategic  decisions  during  all 
phases  of  technology  transition. 

2.  Formulate:  The  team  conducts  an  assessment  using  the  Technology 
Readiness  Level  (TRL)  tool  to  establish  a  baseline  of  the  technology’s 
maturity  and  formulates  a  strategy  to  mature  it  for  a  final  product.  Section 
E  provides  TRL  definitions  and  details. 
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3.  Develop  information:  The  team  gathers  milestone  maturity  infonnation  for 
approval  to  move  to  the  next  level. 

4.  Coordinate  and  update:  The  team  communicates  across  stakeholders  and 
controls  the  approval  process  to  move  to  the  next  gate. 

5.  Commit  and  approve:  The  team  evaluates  and  agrees  to  one  of  the 
decision  criteria,  “go/hold/no-go,”  to  transition  the  technology. 

6.  Transition:  The  likelihood  of  successful  technology  transitions  to  produce 
a  product. 

The  six  gate-stage  phases  are  “intended  to  provide  a  disciplined  approach  to 
executing  a  technology  maturation  effort,  including  consideration  of  all  technical  and 
acquisition  factors  associated  with  a  successful  transition.  Ultimately,  the  use  of  the 
stage-gate  process  will  facilitate  laboratory,  engineering,  and  programmatic  reviews” 
(USAF,  2010,  p.  9).  During  the  Proof  of  Concept  stage,  the  team  approves  the  concept  to 
make  a  prototype.  In  the  Refinement  stage,  the  prototype  is  refined  for  Development  in 
stage-gate  4.  Subsequently,  in  stage-gate  5,  the  developed  prototype  is  tested  in  a 
Relevant  Environment  and  approved  to  transition  to  stage-gate  6.  The  USAF  (2010) 
claimed  that  this  method  formalizes  the  process  and  keeps  their  “eye  on  the  prize.”  In 
general,  this  method  is  a  building  strategy,  beginning  with  R&D  to  mature  the  technology 
and  progress  from  Technology  Readiness  Level  (TRL)  3  through  TRL  6,  then  the  project 
transitions  to  an  acquisition  program  for  further  maturation  (USAF,  2010,  p.  8). 

The  USAF’s  Technology  Transition  model  illustrates  a  revolutionary  process  to 
effectively  and  efficiently  implement  a  concept  into  a  system.  This  section  described  the 
managerial  process  to  effectively  transition  technologies  in  the  DOD.  The  next  section 
explains  how  one  moves  from  processes  to  TRLs. 

E.  TECHNOLOGY  READINESS  LEVEL 

The  DOD  spends  billions  of  dollars  developing  advanced  major  defense 
programs.  Technology  Readiness  Assessment  (TRA)  helps  with  the  evaluation  of  any 
given’s  technology  maturity  in  order  to  reduce  risk  and  cost  of  implementation  and 
utilization  (Department  of  Energy  [DOE],  2012).  The  TRA  is  a  widely  accepted  model 

used  in  the  U.S.  government’s  acquisition  process.  TRL  is  just  one  of  two  approaches  for 
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assessing  technology  maturity.  The  GAO  added  an  additional  element  to  the  evaluation 
process  when  assessing  technology  maturity.  In  addition  to  the  TRA,  the  GAO  (2003) 
presented  a  system’s  profile  element  during  the  maturity  assessment.  The  profile 
includes,  a  general  description,  pictures,  key  elements  of  the  system,  and  a  schedule  that 
covers  milestone  and  completion  dates.  Acquisitions  of  systems  are  not  limited  only  to 
the  DOD:  in  2012,  the  Department  of  Energy  (DOE)  had  over  400  active  projects  such 
that  the  DOE  relied  on  the  TRA  methodology  to  review  and  assess  these  advance  energy 
systems.  Regardless  of  the  agency  or  departments’  assessments  requirement,  the  TRA 
and  system  profile  both  evaluate  the  technology  in  order  to  timely  recognize  and  rectify 
gaps,  mitigate  risk,  control  timelines,  and  reduce  or  control  cost. 

1.  What  Is  Technology  Readiness  Assessment,  and  How  Is  it  Measured? 

The  DOD  (2011)  defined  TRA  as  “a  systematic,  metric-based  process  that 
assesses  the  maturity  of,  and  the  risk  associated  with,  critical  technologies  to  be  used  in 
Major  Defense  Acquisition  Programs  (MDAPs)”  (p.  1-1).  In  order  to  make  the  transition 
from  the  laboratory  to  product,  the  TRA  can  serve  as  a  helpful  knowledge-based  standard 
and  shorthand  used  to  evaluate  the  technology’s  maturity  (ASD[R&E],  2011).  The 
evaluation  provides  a  numeric  value  know  as  Technology  Readiness  Level  (TRL)  that 
measures  technology  from  mature  or  near  mature. 

TRL  is  a  disciplined  ranking  system  of  technology’s  maturity  rated  between  TRL 
1  and  9.  The  TRL  analytical  tool  is  well  defined  and  is  the  best  practice  approach  in 
evaluating  the  technology’s  maturity.  The  milestone  levels  are  as  follow:  TRL  1  contains 
conceptual  documented  paper  studies  of  technology’s  basic  properties;  TRL  2  is  when  the 
concept  becomes  a  practical  application  to  invent  the  system;  TRL  3  is  when  the 
analytical  and  experimental  laboratory  scale  models  are  observed;  TRL  4  undertakes  the 
component  for  validation  in  the  laboratory;  TRL  5  contains  applied  research  laboratory 
scale  experimentation;  TRL  6  is  where  a  model  or  prototype  is  demonstrated  in  a  relevant 
environment;  however,  this  is  not  the  only  consideration  because  the  evaluation  is 
supplemented  with  expert  professional  judgment;  subsequently,  TRL  7,  is  a  prototype 
that  can  be  presented  and  tested  in  actual  relevant  environment;  TRL  8  is  where  the 
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actual  system  is  tested  to  for  demonstration  in  a  plant  environment,  to  consequently  the 
systems  is  built  in  TRL  9,  which  is  ready  for  use  in  operational  mission  conditions 
(ASD[R&E],  2011).  (All  the  TRLs  are  more  fully  explained  below  in  Table  1.)  The 
DOD’s  (2011)  benchmark  is  whether  technological  evolution  gives  greater  capabilities; 
however,  to  reap  the  benefit  and  not  waste  money  during  the  R&D  process,  evaluating 
and  assessing  the  technology’s  maturity  would  increase  efficiency  of  the  investment 
(Sohn  &  Moon,  2003). 

Moreover,  the  GAO  (2003)  assessed  systems  in  two  components:  (1)  a  system’s 
profile  and  (2)  a  product  knowledge  assessment: 

1.  System  profile:  the  first  component  is  the  system  profile  detailing  the 
general  description  of  the  product  with  pictures  and  key  elements  such  as 
schedule  timeline  and  cost  (GAO,  2003). 

2.  Product  knowledge  assessment:  the  second  component  is  the  product 
knowledge  assessment  where  the  technology  is  assesses  and  given  a  TRL 
ranking  (GAO,  2003). 

Knowing  the  levels  provides  an  unbiased  approach  when  evaluating  technology 
maturity.  According  to  the  GAO  (2003;  2009a)  and  DOE  (2012),  the  benchmark  and  best 
practice  to  claim  technology  maturity  is  TRL  7.  Significantly,  TLR  7  enables  the 
technology  prototype  to  be  evaluated  in  a  realistic  operational  environment  resulting  in 
low  risk  reduction  to  enter  production  and  implementation.  The  GAO  (2003;  2009a) 
stated  that  a  system  in  TRL  6  is  referred  as  approaching  or  nearing  maturity  and  is  50% 
of  the  desire  knowledge  level.  In  a  relevant  environment,  technology  is  evaluated  in  a 
high  fidelity  laboratory  simulated  operational  environment,  whereas  an  operational 
environment  is  where  the  system  is  intended  to  operate  (ASD[R&E],  2011).  As  depicted 
in  Figure  3,  the  researcher  drew  from  the  DOE’s  (2012)  framework  for  the  model  flow  of 
technology  assessment  progress  in  order  to  have  a  visual  guide.  Furthermore,  an 
illustration  is  provided  in  Table  1  that  contains  details  of  definition,  description  and 
supporting  information  of  all  the  technology  readiness  levels  in  the  TRL  model. 
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Figure  3.  Progress  Flow  of  Technology  Readiness  Levels  (from  DOE,  2012) 


The  DOE’s  (2012)  TRL  model  was  adapted  to  develop  the  guiding  framework  for 
this  study  as  illustrated  in  Figure  3.  The  process  begins  from  research  department  such  as 
a  university  or  a  research  industry.  Universities  and  industry  research  departments 
continuously  provide  input  throughout  the  cycle  but  the  model  does  not  have  to  include 
this  collaborative  knowledge  exchange  format  and  is  equally  relevant  to  in-house 
development  programs.  Levels  of  readiness  increase  as  concepts  are  moved  to  laboratory 
for  research.  Once  a  prototype  is  developed,  the  prototype  is  tested  in  a  mission 
environment  where  the  technology  is  evaluated.  Full  production  occurs  when  TRL  9  is 
accomplished.  At  each  stage/level  feedback  is  imperative  provided  by  the  project’s 
manager  and  the  project’s  authority  approving  level  in  order  to  control  the  process  and 
meet  requirements. 

Adopted  from  the  USAF  Technology  Development  and  Transition  Strategy 
(TDTS)  Guidebook  (USAF,  2010),  Table  1  provides  a  visual  chart  of  definitions  for  the 
TRL  from  1-9.  Row  1  displays  the  numeric  attributes  for  the  technology  readiness  level, 
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row  2  displays  the  basic  definition  and  row  3  displays  a  detailed  description  for  each 
level.  But,  most  importantly,  a  technology  is  considered  hilly  mature  at  TRL  7,  when  the 
prototype  is  tested  in  the  relevant  operational  environment  and  this  level  is  the  DOD’s 
minimum  maturity  level  for  consideration  to  satisfy  requirements  is  TRL  6  (ASD[R&E], 
2011). 


Table  1 .  Technology  Readiness  Level  Definitions  (horn  USAF,  2010) 


TRL 

Definition 

Description 

Basic  Principles  Observed  and 
Reported 

Lowest  level  of  technology  readiness.  Scientific 
research  begins  translation  to  applied  research  and 
development.  Examples  might  include  paper 
studies  of  a  technology’s  basic  properties. 

2 

Technology  Concept  and/or 
Application  Formulated 

Invention  begins.  Once  basic  principles  are 
observed,  practical  applications  can  be  invented. 
The  application  is  speculative  and  there  is  no  proof 
or  detailed  analysis  to  support  the  assumption. 
Examples  remain  limited  to  paper  studies. 

3 

Analytical  and  Experimental 
Critical  Function  and/or 
Characteristic  Proof  of  Concept 

Active  research  and  development  is  initiated.  This 
includes  analytical  studies  and  laboratory  studies 
to  physically  validate  analytical  predictions  of 
separate  elements  of  the  technology.  Examples 
include  components  that  are  not  yet  integrated  or 
representative. 

4 

Component  and/or  Breadboard 
Validated  in  Laboratory 
Environment 

Basic  technological  components  are  integrated  to 
establish  the  feasibility  that  the  pieces  will  work 
together.  This  is  relatively  low  fidelity  compared 
to  the  eventual  system.  Examples  include 
integration  of  ad  hoc  hardware  in  a  laboratory. 

Component  and/or  Breadboard 
Validated  in  Relevant 
Environment 

Fidelity  of  breadboard  technology  increases 
significantly.  The  basic  technological  components 
are  integrated  with  reasonably  realistic  supporting 
elements  so  that  the  technology  can  be  tested  in  a 
simulated  environment.  Examples  include  high- 
fidelity  laboratory  integration  of  components. 

TRL 

Definition 

Description 
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6 

System/Subsystem  Model  or 
Prototype  Demonstrated  in 
Relevant  Environment 

Representative  model  or  prototype  system  well 
beyond  the  breadboard  tested  for  TRL  5  undergoes 
test  in  a  relevant  environment.  Represents  a  major- 
step  in  a  technology’s  demonstrated  readiness. 
Examples  include  prototype  testing  in  a  high- 
fidelity  laboratory  environment  or  simulated 
operational  environment. 

7 

System  Prototype  Demonstrated 
in  Operational  Environment 

Prototype  near  or  at  planned  operational  system. 
Represents  a  major  advance  from  TRL  6  and 
requires  demonstration  of  an  actual  system 
prototype  in  an  operational  environment  (e.g.,  in 
an  aircraft  or  other  vehicle  or  in  space).  Examples 
include  prototype  testing  in  a  test  bed  aircraft. 

8 

Actual  System  Completed  and 
Flight-Qualified  Through  Test 
and  Demonstration 

Technology  has  been  proven  to  work  in  its  final 
form  and  under  expected  conditions.  In  ahnost  all 
cases,  this  TRL  represents  the  end  of  true  system 
development.  Examples  include  developmental  test 
and  evaluation  of  the  system  in  its  intended 
weapon  system  to  determine  whether  it  meets 
design  specifications. 

9 

Actual  System  Flight-Proven 
Through  Successful  Mission 
Operations 

Actual  application  of  the  technology  in  its  final 
form  and  under  mission  conditions,  such  as  those 
encountered  in  operational  test  and  evaluation.  In 
ahnost  all  cases,  this  ends  final  debugging  aspects 
of  true  system  development.  Examples  include 
system  usage  under  operational  mission 
conditions. 

In  summary,  this  section  provided  a  general  discussion  of  the  label  system  when 
ranking  technology’s  maturity  ranging  between  TRLs  1-9.  Since  the  creation  of  the 
technology  government,  companies  have  used  it  increase  competiveness  by  having 
policies  in  place  to  achieve  successful  products  and  systems  development.  The  next 
section  discusses  what  the  U.S.  government  and  the  DOD  have  done  to  guide  innovation. 

F.  FEDERAL  ACQUISITION  STATUE  AND  LAW,  AND  DOD  DIRECTIVE 
AND  INSTRUCTION 

Over  the  past  three  decades,  the  U.S.  government  has  developed  an  increased 
interest  in  regulating  all  acquisitions  of  supplies  and  seivices  by  all  the  federal  executive 
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agencies.  In  addition,  the  federal  government  issued  laws  to  regulate  the  technology 
transfer  from  the  commercial  sector  to  use  in  the  DOD.  This  section  reviews  the  Federal 
Acquisition  Regulation  (FAR),  major  DOD  policies  and  directives  related  to  regulating 
technology  transition,  and  the  risks  and  benefits  associated  with  the  acquisitions  of 
commercial  off-the-shelf  items  (COTS)  to  support  the  mission. 

1.  Federal  Acquisition  Regulation 

U.S.  government’s  acquisition  regulation  became  effective  in  April  1984  (FAR, 
2004).  The  government’s  preference  for  procurement  of  commercial  items  is  contained  in 
the  following  statutes:  41  U.S.C.  §  1906,  §  1907,  and  §  3307  and  10  U.S.C.  §  2375-2377 
(FAR,  2004,  p.  12.1-1).  In  addition,  the  acquisition  approach  for  the  technology 
maturation  effort  of  systems  is  covered  in:  10  U.S.C.  §  2304,  10  U.S.C.  §  2305,  and  10 
U.S.C.  §  2306)  (USAF,  2010,  p25).  The  FAR  (2004)  is  the  strategic  policy  that  directs 
agencies  to  “conduct  market  research  to  detennine  whether  commercial  items  or  non- 
developmental  items  are  available  that  could  meet  the  agency’s  requirements”  (p.  12.1- 
1).  The  foundation  to  link  the  private  sector’s  technological  leadership  with  the  DOD’s 
developmental  technological  needs  was  emphasized  in  the  2001  Quadrennial  Defense 
Review  Report  in  order  to  provide  the  best  piece  support  in  the  field  (OSD,  2001).  The 
FAR  is  the  documented  source  that  guides  DOD  towards  integration  of  the  commercial 
sector’s  capabilities  to  meet  the  U.S.  military’s  technological  needs,  while  the  DOD 
directives  and  instructions  provide  the  template  to  managing  and  acquiring  COTS. 

2.  Department  of  Defense  Directives 

The  DOD  annually  spends  billions  of  taxpayer  dollars  developing  and  procuring 
systems.  In  order  to  ensure  positive  returns  on  investment,  the  DOD  has  established 
policies  and  directives  to  adopt  the  “best  practice”  in  developing  and  procuring 
equipment  and  weapons  quickly  and  cheaply  (GAO,  2003)  The  DOD’s  best  practice  to 
manage  the  procurement  of  systems  and  technology  transition  procedures  are  underlined 
by  the  mandatory  principles,  DOD  Directive  (DODD)  5000.1,  The  Defense  Acquisition 
System;  and  DOD  Instruction  (DODI)  5000.2  Interim,  The  Operation  of  the  Defense 
Acquisition  Systems. 
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a. 


DOD  Directive  5000.1 


Congress  and  DOD  leaders  recognize  the  contribution  and  benefit  associated  with 
the  procurement  of  COTS  products,  services,  and  technology  for  greater  effectiveness 
and  efficiency.  One  of  the  primary  objectives  of  DODD  5000.1  (DOD,  2007)  is  to 
promote  innovation,  flexibility  and  management.  The  policy’s  objective  is  “to  acquire 
quality  products  that  satisfy  user  needs  with  measurable  improvements  to  mission 
capability  and  operational  support,  in  a  timely  manner,  and  at  a  fair  and  reasonable  price” 
(DOD,  2007,  p.  3).  The  DODD  5000.1  (DOD,  2007)  stated  the  following  with  regard  to 
market  research  and  procurement: 

DOD  Component s)  shall  consider  multiple  concepts  and  analyze  possible 
alternative  ways  to  satisfy  the  user  need.  ...The  DOD  Components  shall 
seek  the  most  cost-effective  solution  over  the  system’s  life  cycle.  They 
shall  conduct  market  research  and  analysis  to  detennine  the  availability, 
suitability,  operational  supportability,  interoperability,  safety,  and  ease  of 
integration  of  the  considered  and  selected  procurement  solutions.  The 
DOD  Components  shall  work  with  users  to  define  capability  needs  that 
facilitate  the  following,  listed  in  descending  order  of  preference: 

The  procurement  or  modification  of  commercially  available  products, 
services,  and  technologies,  from  domestic  or  international  sources,  or  the 
development  of  dual-use  technologies,  (p.  8) 

DODD  5000.1,  The  Defense  Acquisition  System,  describes  the  management  principles  for 
DOD’s  knowledge-based  acquisition  programs  and  these  principles  are  a  mandated 
directive  that  stresses  best  practice  approach  during  the  development  of  systems  process 
(GAO,  2003). 

b.  DOD  Instruction  5000.2  Interim 

An  important  task  during  the  conduct  of  technology  transition  is  the  enhancement 
or  rapid  development  of  military  systems.  The  primary  element  involved  during  the 
system  engineering  processes  is  to  “mature,  prototype,  and  demonstrate  the  suite  of 
selected  system  elements  and  complete  the  preliminary  design  of  the  full  system  for  low- 
risk  entry  to  Engineering  and  Manufacturing  Development”  (DOD,  2013a,  p.  232).  To 
promote  the  technology  transition  and  maturation  the  DOD  Instruction  5000.2  Interim 

(DOD,  2013a)  provides  the  following  amplifying  guidance: 
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...an  investment  decision  to  pursue  specific  product  or  design  concepts, 
and  to  commit  the  resources  required  to  mature  technology  and/or  reduce 
any  risks  that  must  be  mitigated  prior  to  decisions  committing  the 
resources  needed  for  development  leading  to  production  and  fielding. 

(P-  V) 

Furthermore,  the  directive  provides  detailed  guidance  with  regard  to  acquiring  COTS 
from  the  commercial  sector  such  as  small  business,  academia  and  government 
laboratories: 

Promising  technologies  shall  be  identified  from  all  sources  domestic  and 
foreign,  including  government  laboratories  and  centers,  academia,  and  the 
commercial  sector.  In  addition,  PMs  [program  managers]  shall  consider 
the  use  of  technologies  developed  under  the  Small  Business  Innovation 
Research  (SBIR)  program,  and  give  favorable  consideration  to  successful 
SBIR  technologies.  The  risk  of  introducing  these  technologies  into  the 
acquisition  process  shall  be  reduced;  coordination,  cooperation,  and 
mutual  understanding  of  technology  issues  shall  be  promoted.  The  conduct 
of  Science  and  Technology  (S&T)  activities  shall  not  preclude,  and  where 
practicable,  shall  facilitate  future  competition.  (DOD,  2013a,  p.  14) 

DOD  Instruction  5000.2  Interim,  The  Operation  of  the  Defense  Acquisition  System, 
outlines  a  framework  for  managing  acquisition  programs.  According  the  GAO  (2013), 
about  40%  of  all  2012  systems  were  25%  or  more  over  budget  with  over  two  years  of 
scheduled  delays.  To  provide  control  and  improvement  of  these  two  elements — cost  and 
schedule — the  updated  DODD  is  expected  to  provide  flexibility  to  procure  COTS.  The 
directive  indicates,  with  great  emphasis,  that  there  is  a  relationship  between  requirements, 
acquisitions,  and  budgeting  processes  such  that  all  three  components  work  together  to 
identify  and  resolve  issues  early  as  possible  (DOD,  2013a).  Moreover,  the  directive 
suggests  that  actions  required  for  cost  reduction,  especially  for  information  technology 
systems,  is  achieved  during  the  compliance  to  “redesign  the  processes  that  the  system 
supports  to  reduce  costs,  improve  effectiveness  and  maximize  the  use  of  commercial  off- 
the-shelf  technology”  (DOD,  2013a,  p.  71). 

c.  Commercial-Off-the-Shelf  Regulations 

The  FAR  and  DOD  regulation  encourages  the  acquisitions  of  commercial-off-the 
shelf  items.  According  to  Allen  (2008),  the  use  of  COTS  involves  accepting  some 
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benefits  and  risks.  However,  the  according  to  the  DOD  Instruction  5000.2,  the  use  of 
COTS  is  worth  investigating: 


The  use  of  commercial-off-the-shelf  (COTS)  items,  including  Non- 
Developmental  Items,  can  provide  significant  opportunities  for 
efficiencies  during  system  development  but  also  can  introduce  certain 
issues  that  should  be  considered  and  mitigated  if  the  program  is  to  realize 
the  expected  benefits.  (DOD,  2013a,  p.  175) 

According  to  the  directive  (DOD,  2013a)  the  benefits  associated  with  COTS  include  the 
following: 

1 .  System  development  time  reduction 

2.  A  faster  integration  of  technology 

3.  Lower  system  life-cycle  cost  is  lower  due  to  leveraging  available 
commercial  technology 

Nevertheless,  there  are  risks  associated  with  using  COTS.  Allen  (2008)  put  it  best 
when  specifically  referring  to  the  procuring  of  software  COTS:  stating  “the  risk  is  that 
selecting  the  wrong  COTS  software  component  might  be  more  expensive  than  fixing 
problems  in  custom-built  software”  (p.  80).  Furthermore,  according  to  the  directive 
(DOD,  2013a),  there  are  concerns  associated  with  COTS.  The  following,  relevant  to  this 
study,  are  five  major  elements  to  name  just  a  few  a  from  wide-ranging  list: 

1.  “If  integration  requires  a  ‘modified  COTS  product,’  meaning  that  a  COTS 
product  may  not  be  designed  for  many  military  environments  (which,  by 
definition,  is  not  a  COTS  product  under  section  403  of  title  41,  United 
States  Code,  but  is  allowed  under  section  43 1  of  title  4 1 ,  United  States 
Code),  then  the  program  may  lose  the  ability  to  use  the  vendor’s 
subsequent  product  upgrades  or  to  find  a  suitable  replacement  for  the 
product  from  other  commercial  sources”  (DOD,  2013a,  p.  175). 

2.  “The  vendors  can  embed  proprietary  functions  into  COTS,  limiting  supply 
sources”  (DOD,  2013a,  p.  175). 

3.  Supply  chain  risk  management  of  COTS  items  is  limited  by  the  vendor, 
who  is  under  no  obligation  to  the  purchaser  to  provide  such  information 
(DOD,  2013a,  p.  175). 

4.  Difficulty  in  finding  suitable  replacements  and/or  alternate  items  if  the 
COTS  vendor  stops  manufacturing  the  product  or  changes  the 
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configuration  drastically,  requiring  the  need  to  maintain  different 
configurations  of  a  single  product  (DOD,  2013a,  p.  175). 

5.  The  graphical  user  interface  (GUI)  design  may  not  completely  support 

user  tasks,  which  can  cause  inefficient  workarounds  and  improper  use  of 
the  system  by  the  user  (DOD,  2013a,  p.  175). 

According  to  Allen  (2008),  the  best  approach  to  reducing  risk  and  concern  when 
utilizing  COTS  is  to  adhere  to  DOD  policies,  maintain  flexibility  when  incorporating 
COTS  into  the  project,  avoid  restrictive  statements  of  work  and  avoid  very  specific 
details  that  the  technology  ends  up  being  unique  to  government  (p.  80).  Regardless  of  the 
extent  to  which  the  agency  plans  to  use  COTS,  rigorous  testing,  evaluation,  and 
acceptance  of  some  risk  must  be  undertaken  in  order  to  integrate  the  technology, 
streamline  process,  and  reduce  cost. 

This  section  introduced  statutory  preferences,  directives  and  regulations  that 
authorize  the  DOD  to  consider  procurement  of  technological  capabilities  from  the 
commercial  sector.  The  core  mission  is  for  the  DOD  to  consider  commercial 
technological  capabilities  during  the  decision  processes  to  possibly  reduce  development 
time,  cost  and  leverage  technology  to  sustain  a  competitive  edge.  Today,  these  laws  and 
guidance  dominate  the  DOD’s  nature  of  acquiring  major  weapon  systems  in  order  to 
reduce  cost  (Markman  et  ah,  2008) 

G.  COTS  OBJECT/SHAPE  RECOGNITION  TECHNOLOGY 

In  day-to-day  routine,  humans  rely  heavily  on  their  vision  to  identify  the  world  as 
they  see  it.  This  is  a  natural  and  effortless  human  process;  however,  to  develop  a  machine 
to  accomplish  the  process  is  a  complex  and  ambitious  project.  Currently,  a  group  of 
scientists  has  developed  a  prototype  of  object/shape  recognition  and  images  processing 
prototype  that  can  identify  leafs  (Rajpurohit,  Bhat,  Devurkar,  Patil,  &  Sirbi,  2013). 
Scientists  are  rapidly  improving  and  advancing  the  technology  and  some  major 
companies  have  adopted  early  versions.  This  section  describes  object/shape  recognition 
technology  and  its  utility  in  the  private  and  government  sector. 


23 


1.  What  Is  Object/shape  Recognition  Technology 

Object/shape  recognition  technology  is  a  tool  that  identifies  items  using  a  vision¬ 
capturing  device  and  recognizes  an  object  as  a  member  of  a  category  containing  other 
similar  objects  (Frome,  1996).  The  application  of  the  object/shape  recognition  technique 
by  an  image-capturing  tool  is  an  approach  to  identify  items  for  the  purpose  of 
classification  (Rajpurohit  et  ah,  2013).  There  are  different  approaches  and  methods,  but 
all  have  elements  in  common:  algorithm  program,  image  capture  apparatus  and  database 
to  compare  the  image  of  interest  to  provide  feedback. 

Object/shape  recognition  technology  uses  a  program  to  index  a  database  in  order 
to  translate  received  data,  and  then  compares  it  to  data  in  the  database  to  provide 
feedback.  In  general,  the  infrastructure  consists  of  several  components  that  include  an 
image  capture  apparatus  for  shape  searching.  The  image  signature  is  turned  into  digital 
data  in  order  to  compare  potential  leads  in  the  database.  The  database  server  contains  all 
information  for  matching  as  well  as  adding  and  classifying  newly  captured  data.  Another 
added  feature  would  be  that  the  program  learns  while  comparing,  so  its  information  is 
readily  available  for  future  use.  Finally,  the  tool  returns  feedback  through  wireless  or 
cable  connection.  Conceptually,  the  tool  identifies  the  object  and  its  shape;  then  it 
translates  that  image  into  algorithm  to  digitally  search  a  like  category  from  the  database; 
and  finally,  it  provides  feedback  with  the  matching  item  (David  &  DeMenthon,  2006). 

2.  Object/shape  Recognition  Technology’s  Utility  in  Private  and 
Government  Sectors 

Many  major  companies  in  the  private  sector  and  U.S.  government  are  using  some 
form  of  object/shape  recognition  technology  to  differentiate  and  carve  out  a  competitive 
edge,  for  instance,  Amazon,  with  its  newly  released  Fire  Phone  that  has  an  application 
called  Firefly.  The  Firefly  application  captures  images  of  items  for  the  purpose  searching 
it  on  Amazon.com  for  purchase  (Matz,  2014).  Similarly,  the  DOD  has  an  invested 
interest  in  the  technology’s  benefit  such  that  the  Naval  Research  Lab  (NRL)  has  created  a 
prototype. 
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Computer  vision,  image  processing  detection  and  recognition,  and  machine 
learning  have  increasingly  become  of  great  interest  to  the  DOD  for  general  intelligence, 
target  recognition,  and  data  analysis.  Currently,  the  NRL  claims  to  have  developed  a 
“rapid  recognition  and  location  of  surface  shapes  in  range  images”  (Manak,  2014). 
According  to  the  NRL,  the  technology  can  differentiate  parts  in  a  clutter  environment 
with  accurate  recognition. 

Accordingly,  the  object/shape  recognition  technology  from  Amazon  and  NRL  are 
two  examples  from  the  private  and  government  sector.  The  U.S.  government  and  private 
sectors’  interest  in  recognition  technology  is  for  enhancement  in  their  internal  processes’ 
efficiency  or  to  carve  a  competitive  edge.  Their  interest  in  the  benefits  attained  by  the 
technology  ensures  that  gaps  are  filled  to  evolve  the  object/shape  recognition  technology. 
Additionally,  with  the  advancement  of  new  commercial  technology  and  resourceful 
databases,  there  are  opportunities  to  mature  object/shape  recognition  technology. 

H.  CHAPTER  SUMMARY 

With  technology  rapidly  evolving  and  the  increasing  need  to  sustain  a  competitive 
edge,  the  federal  government  must  keep  abreast  of  technological  changes.  It  is  beneficial 
to  ensure  that  both  the  government  and  the  industrial  sector  maintain  open  lines  of 
communication  in  order  to  collaborate  and  meet  efficiency  requirements.  This  chapter 
reviewed  the  elements  that  may  impact  the  decision-making  with  regard  to  improving  the 
current  property  processing  systems.  Specifically,  this  chapter  reviewed  the  generally 
accepted  processes  for  technology  commercialization,  CBA,  U.S.  government’s 
assessment  of  technology  transition,  technology  readiness  and,  statutory  policies, 
directives  and  regulations. 

The  take  away  is  that  TC  differentiates  the  technology  to  target  the  appropriate 
consumers  or  users.  Parameters  or  measuring  techniques  are  in  place  to  reduce  risk, 
control  cost,  and  creates  timeline.  The  evolutionary  process  begins  from  concept, 
continues  onto  the  lab  for  development,  and  ends  with  an  operational  product  for 
warfighters  to  use  in  the  actual  field  environment.  The  TT  process  is  one  of  the  main 
frameworks  that  effectively  incorporates  the  system  or  product.  TRLs  establish 
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parameters  to  evaluate  technology’s  maturity.  In  general,  a  technology  is  claimed  to  be 
mature  for  production,  or  roll-out  in  the  case  of  software,  when  it  accomplishes  TRL  7. 
However,  DOD  considers  TRL  6  the  benchmark  to  proceed  with  system  development. 
The  statutory  codes  and  DOD  guidance  provide  direct  orders  to  follow  with  regard  to 
considering  the  commercial  industry’s  technological  advancement  and  adapting  it  to  the 
government’s  needs.  All  the  theories,  guidance,  and  concepts  reviewed  in  this  chapter  are 
important  elements  to  refer  to  and  which  guided  this  study  in  answering  the  research 
questions.  The  focus  of  the  subsequent  chapter  is  to  describe  the  research  approach  and 
methodology  that  lays  the  foundation  of  the  analyses  for  conducting  this  study. 
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III.  METHODOLOGY 


A.  INTRODUCTION 

This  chapter  presents  in  detail  the  analysis  steps  taken  to  address  this  project’s 
objectives,  to  conduct  an  analysis  of  the  DLA  DS  property  process  and  evaluate  the 
technology  readiness  level  (TRL)  in  order  to  assess  the  feasibility  to  implementing  a 
property  identification  technology,  and  to  conduct  a  cost-benefit  analysis  (CBA).  This 
applied  research  study  utilizes  mixed  methods.  Mixed  methods  are  used  to  conduct  three 
analyses.  First,  the  study  uses  a  process  analysis  to  understand  DLA  DS’s  property 
disposal  operational  process.  Second,  it  also  uses  a  technology  readiness  assessment 
method  to  assess  the  viability  of  implementing  object/shape  recognition  technology  by 
the  DLA  DS.  Finally,  the  study  utilizes  a  CBA  method  in  order  to  assess  DLA  DS’ 
property  identification  operational  cost  and  benefit  values  provided  with  technology  and 
without  technology  for  the  purpose  of  financial  analysis.  Each  method  and  its  respective 
data  sources  for  each  are  described  in  the  following  sections.  The  approach’s  purpose  is 
to  set  a  baseline  to  conduct  the  qualitative  and  quantitative  analysis. 

The  main  objective  of  an  applied  research  is  to  explore  a  practical  situation’s 
solution  to  a  problem  (Booth,  Colomb,  &  Williams,  2008).  The  applied  research  has  two 
major  components.  One  component  is  knowing  how  to  account  for  factors  in  identifying 
the  problem;  the  second  is  doing — rectifying  the  problem  with  a  solution  (Booth  et  ah, 
2008).  This  thesis’  focus  of  effort  is  to  find  a  solution  to  the  DLA  DS  property 
identification  process  determine  feasibility  in  order  to  apply  technology  to  the  process 
therefore;  for  this  reason  the  researcher  designed  this  applied  research  methodology. 

B.  PROPERTY  DISPOSAL  PROCESS  ANALYSIS  METHODOLOGY  STEPS 

This  section  underlines  the  steps  by  which  the  researcher  analyzed  the  property 
identification  process.  Currently,  DLA  DS  disposes  of  items  using  a  seven-step  process. 
Each  step  was  analyzed  to  understand  the  process  flow  and  to  identify  inefficiencies  and 
problems. 
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1. 


Data 


The  researcher  visited  DLA  DS  at  Camp  Pendleton  to  observe  procedures  for 
processing  property.  The  researcher  visited  the  site  and  observed  the  process  for  three 
hours  and  asked  clarifying  questions  to  document  the  process.  During  the  visit  the 
researcher  took  detailed  notes  and  pictures  and  identified  relevant  documents  such  as 
standard  operating  procedures,  training  manuals  and  reports  from  the  Government 
Accountability  Office  (GAO).  These  reports  were  then  collected  to  provide  data  for  the 
study. 


a.  Data  from  Previous  Studies 

In  2008,  DLA  DS  captured  investigated  DS  property  process.  During  that  study 
one  of  DLA  DS  facilities  was  observed  processing  property  (see  Figure  4;  DLA  DS, 
2008).  The  result  from  the  2008  study  is  particularly  important  because  it  provides  a 
sample  number  and  the  time  it  took  to  accomplish  each  task.  Steps  of  focus  for  this  thesis 
are  5,  6,  and  7.  Step  5  contains  the  sample  size  of  items  25  items’  local  stock  numbers 
(LSNs).  Steps  6  and  7  contain  the  time  it  took  to  match  and  research  LSN’s.  Although 
plenty  of  analysis  can  be  generated  from  the  “time  study”  data,  the  interest  of  this  thesis 
is  to  calculate  the  average  time  to  research  each  LSN  item  that  needed  further 
verification;  therefore,  the  following  represents  the  generic  average  formula: 
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Figure  4.  Process  Flow  Time  Study  per  Step  (from  DLA  DS,  2008) 


Note:  Average  researched  time  to  per  LSN  =  Total  time  to  research  LSN’s  /  total  number 
of  LSN’s  researched 

Depicted  in  Figure  4  is  the  average  time  employees  take  during  each  step  for  one 
cycle  of  property  processing.  A  description  of  the  columns  1  through  5  is  as  follows: 
responsible  employee  completing  the  task,  step  in  the  process,  time  to  accomplish  task  in 
the  respective  step,  description  of  task  process  undertaken  by  employee,  and  notes 
identified  during  the  study,  respectively.  The  carrying  time  weight  per  step  in  Steps  5,  6 
and  7  is  notable.  Step  5  takes  the  longest  to  accomplish  while  Step  6  carries  the  third- 
most  time  in  the  cycle. 

The  most  recent  study  was  conducted  in  2013.  The  study  resulted  in  streamlining 
the  property  process  by  up  35%  (see  Figure  5;  DLA  DS,  2013).  This  information  is 
particularly  relevant  to  this  thesis  because  it  provides  the  standard  by  which  employees’ 


29 


performance  is  measured.  The  measurement  relevant  for  this  thesis  is  the  final  average 
resulting  in  7.5  receipts  per  person  per  hour.  These  results  were  utilized  to  calculate  hours 
spent  processing  items  in  the  item  processing  analysis. 


Receipt  Process  Data  (Property  Entering  CDD  gate  to  Property  Stowed) 


(1)  Day 
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.'.Average  Receipts  per  Person  per  Hour:  7.5 


Figure  5.  Study  to  Measure  per  Person  Hours  per  Property  Received  for  Process 

(from  DLADS,  2013) 

Figure  5  illustrates  in  a  four-week  period  the  average  receipts  of  each  person  per 
hour,  7.5.  Column  1  indicates  the  day  of  the  week  while  column  2  is  the  date;  column  3 
illustrates  the  number  of  trucks  for  the  day;  column  4  records  the  full-time  equivalent 
(FTE)  and/or  people  doing  the  work  who  are  on  temporary  duty  assignment  (TDY); 
column  5  shows  how  many  items  were  received,  and  column  6  is  a  ratio  of  items  received 
divided  by  FTE  received  (col  5/col4;  DLA  DS,  2013). 

b.  New  Data,  Collected  from  DLA  DS  at  Camp  Pendleton 

The  author  of  this  thesis  travelled  to  Camp  Pendleton,  CA,  and  collected  two 
types  of  data  sets  that  cover  four  months  of  property  processing  information.  One  data  set 
contains  data  on  the  total  amount  of  unidentified  items  that  were  researched  during  the 
week.  The  second  data  set  contains  daily,  weekly,  monthly  and  annually  data  on 
identified  (input)  and  processed  (output)  items  at  Camp  Pendleton  staring  with  date  1 
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July  2014  until  date  21  October  2014.  The  summaries  of  the  two  data  sets  are  presented 
in  Figures  6  and  7,  respectively.  The  main  take  away  from  Figure  6  is  the  total  number  of 
researched  LSNs  that  were  reported  on  July  14  used  to  calculate  a  historical  quantitative 
relation  between  items  researched  and  items  received.  This  calculation  of  the  ratio 
number  is  presented  in  Figure  7.  The  generic  ratio  formula  is  as  follows: 
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D 
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A 

Disp.  Svcs  Site: 
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The  LSN(’s)  listed  on  this  form  have  been  Researched  and  Demil  Code  Verified  as 
indicated  above.  The  LSN(s)  above  have  been  authorized  for  release  IAW  DLA 
Dispo  Svcs  Policy. 

Printed  CPB  Certifiers  Name  CPB  Certifier  Signature 

Printed  CPB  Verifiers  Name  CPB  Verifier  Signature 


Note:  Please  make  sure  you  are  reviewing  Special  Processing  Requirements  for  item(s)  that  may  require  Special  Processing. 
It  is  each  Field  Sites  responsibility  to  ensure  special  processing  requirements  are  met 
Items  that  are  not  able  to  be  Verified  by  CPB  shall  be  downgraded  according  to  Policy. 


Figure  6.  Researched  Local  Stock  Number,  7/14/14  (from  DLA  DS,  2014) 


Note:  Ratio  of  LSN  items  researched  in  a  week  compared  to  total  items  received  =  Total 
LSN’s  researched  (sum  LSN  in  Figure  6)  /  total  number  of  receipts  for  the  week  (weekly 
sum  of  receipts  from  Figure  7) 


31 


A 

B 

- 

D 

E 

F 

G 

H 

1 

J 

1 

Jul 

through — 

7/31/14 

sorted  by  Region,  Site  &  Date 

Point  in 

Time 

RCN  Open 

Point  in 

Time 

RCN  AGE 

RCN  to 

Induct 

(RCNs) 

Point  in  Time 

Backlog  Ending 
(Lines) 

Point 

in  Time 

tory 

(Lines) 

ADO  New  DAILY 

Data  to  bottom 
{Paste  Values), 
then  Sort  by 

Region,  Site,  Date 

Rece 

(Line 

Bate 

pts 

+  inven 

ties)  Level 

Date 

Region 

Site  ID 

Site 

Count 

Oldest 

RCN 

Days  Avg 

Backlog  Count 

Receipts  Inventory 

7/2/14 

West 

RCCP 

DLA  DS  PENDLETON 

30 

12 

8.6 

1,104 

256 1| 

3.818 

7/3/14 

West 

RCCP 

DLA  DS  PENDLETON 

30 

12 

8  6 

- 

7/4/14 

West 

RCCP 

DLA  DS  PENDLETON 

30 

12 

0 

7/5/14  West 

RCCP 

DLA  DS  PENDLETON 

30 

12 

to_ L 

7/6/14 

West 

RCCP 

DLA  DS  PENDLETON 

« 

1*1 

- 

L703 

0 

7/7/14  West 

RCCP 

DLA  DS  PENDLETON 

31 

iiir 

9.9]|  2,320 

233.  || 

3,812 

7/8/14  West 

RCCP 

DLA  DS  PENDLETON 

31 

14 

10.0 

2,417 

357 1| 

3,487 

7/9/14  West 

RCCP  1DLA  DS  PENDLETON 

34 

13 

7.9 

_ 

311 1| 

3,508 

7/10/14 

West 

RCCP 

DLA  DS  PENDLETON 

37 

14 

3.6 

2,530 

298|| 

3,539 

7/11/14  West 

RCCP 

DLA  DS  PENDLETON 

37 

14 

8.5 

- 

273H- 

7/12/14 

West 

RCCP 

DLA  DS  PENDLETON 

57 

14 

- 

7/13/14  West 

RCCP 

DLA  DS  PENDLETON 

40 

3 'FI 

2,538 

3.568] 

7/14/14 

West 

RCCP 

DLA  DS  PENDLETON 

44 

33; 

i4  r 

2.353* 

351 

33991 

7/15/14  West 

RCCP 

DLA  DS  PENDLETON 

«ZJH 

ie|| 

12.5 

2,108 

333 1 

3,477 

7/16/14 

West 

RCCP 

DLA  DS  PENDLETON 

51 

37 

13.5 

2.055 

303|| 

3,452 

7/17/14  West 

RCCP 

DLA  DS  PENDLETON 

18 

12.5 

2,099 

301|| 

3,014 

7/18/14 

West 

RCCP 

DLA  DS  PENDLETON 

33 

18 

10.5 

- 

737  N- 

7/19/14  West 

RCCP 

DLA  DS  PENDLETON 

51 

18 

16.1 

401  f- 

7/20/14  West 

RCCP 

DLA  DS  PENDLETON 

45 

' 

i3" 

3.119| 

7/21/14  West 

RCCP 

DLA  DS  PENDLETON 

1ZJ[ 

M|| 

IS  4||  2,446 

157|| 

3,096 

7/22/14  West 

RCCP 

DLA  DS  PENDLETON 

54 

.6 

13.4 

2.431H 

213  U 

3,023 1 

7/23/14  West 

RCCP 

DLA  DS  PENDLETON 

59 

37 

6.4 

743°1  

>°7li 

2,993 

7/24/14 

West 

RCCP 

DLA  DS  PENDLETON 

33 

18 

4.9 

2.273 

230 1| 

2,981 

7/25/14  West 

RCCP 

DLA  DS  PENDLETON 

33 

18 

5.4 

- 

- 30T|p - 

7/26/14 

West 

RCCP 

DLA  DS  PENDLETON 

sTi  187—  I- 

? Jb 

7/27/14  West 

RCCP 

DLA  DS  PENDLETON 

52 

71 

- 

2.456 

2,918] 

7/28/14 

West 

RCCP 

DLA  DS  PENDLETON 

47 

77 

10.2 

2,313 

508|! 

2,871 

7/29/14  West 

RCCP 

DLA  DS  PENDLETON 

49 

73 

13.7 

2.383 

437|| 

2,877 

7/30/14 

West 

RCCP 

DLA  DS  PENDLETON 

33 

24 

6.9||  2,601j| 

520|| 

2,897 

7/31/14  West 

RCCP 

DLA  DS  PENDLETON 

62 

73 

13.7 1|  3.072] 

140  |l 

2,927 

Ratio  of  LSN  items  researched  in  a  month  compared  to  total  items  received 

RATIO 

' 

0.0075  jf  2.261  If 

0.054 

Figure  7.  DLA  DS  at  Camp  Pendleton  Daily  Counts  of  Inventory  Processed  during 

July  2014  (from  DLA  DS,  2014) 


Data  presented  in  Figure  7  was  analyzed  to  determine  the  effects  of  the  Receipt 
Control  Number  (RCN)  on  input.  Open  RCN,  RCN  age,  RCN  to  induct,  ending  backlog, 
running  receipts  and  ending  inventory  are  illustrated  in  columns  E-J,  respectively.  Open 
RCN  accounts  for  RCNs  inducted  to  day  and  are  being  tracked.  Oldest  RCN  indicated 
days  oldest  RCN  aged;  items  are  processed  using  the  first-in,  first-out  method.  RCN  to 
induct  indicates  the  average  amount  of  days  to  process  an  RCN;  per  guidance  it  should  be 
processed  in  three  days.  Backlog  ending,  reports  an  estimate  amount  of  items  that  are 
stowed  for  future  process.  Receipts,  reports  total  amount  of  items  processed  for  the  day. 
Inventory  level  refers  to  total  amount  of  items  in  the  warehouse  that  have  been  processed 
and  are  usable.  The  interest  of  this  thesis  is  to  analyze  item  input  and  objectively  assess 
the  output.  Moreover,  for  this  thesis,  the  amounts  of  employees  are  considered  with 
regard  to  average  per  employee  per  hour  that  can  processing  items  for  disposal,  thus  the 
number  of  employees  that  are  employed  to  process  items  impact  the  output.  The 
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remaining  items  will  be  considered  to  be  backlog  for  future  processing.  The  coefficients 
are  explained  by  the  simple  fonnula: 

backlog  =  input  -  output 

To  illustrate  backlog  trend  a  broad  understanding  of  the  sales  backlog  formula 
was  adapted  and  modified  to  illustrate  items  backlog  rather  than  sales  backlog  (Bragg, 
2012).  The  researcher  sought  to  identify  items’  backlog  worth  of  work  days  for  a  crew  of 
six  employees,  therefore  the  sales  formula  was  modified  to  fit  DLA  DS  property  backlog. 
The  result  is  related  to  the  rate  of  the  estimated  number  of  items  received  then  stowed, 
and  the  rate  of  items  processed.  So,  the  backlog  is  the  number  of  items  that  remain 
unprocessed  in  a  time  span.  The  formula  is  as  follows: 

Days'  worth  of  backlog  = - Back'°g  i,ems - 

Processed  items  per  day 

The  ratio  considers  the  number  of  items  backlog  divided  by  the  number  of  items 
processed  (receipts)  per  day.  If  the  number  of  backlogged  items  is  greater  than  the 
number  of  items  processed  (receipts),  then  the  ratio  will  be  greater  than  1  day.  This 
indicates  that  the  facility  receives  more  items  and  cannot  keep  up  with  more  than  the 
demand  than  it  processed.  Conversely,  if  the  number  of  backlog  items  is  less  than  the 
number  of  items  processed  (receipts),  the  ratio  will  be  less  than  1  day.  This  indicates  that 
the  facility  can  keep  up  with  demand  to  process  as  many  items  as  it  receives. 

Both  data  sets  provided  by  DLA  DS  have  relevance  to  this  study  because  they 
contain  metric  measurements  captured  on  a  daily  operational  basis.  Therefore,  the  data 
analysis  provides  meaningful  empirical  evidence  to  compare  during  the  quantitative  and 
financial  analysis  for  the  CBA. 

c.  Data  Normalization  Procedure 

One  of  the  most  challenging  tasks  when  collecting  primary  raw  data  is  the  data 
nonnalization  process,  because  one  needs  to  be  careful  not  to  adjust  the  data  and  change 
its  empirical  value.  In  order  to  attain  better  approximation  of  time-task-duration  the 

33 


researcher  drew  from  the  Program  Evaluation  and  Review  Technique  (PERT)  and  the 
Pearson  Tukey  formula  to  conduct  data  normalization.  PERT  estimates  time  in  an 
uncertainty  scenario  and  to  best  define  the  time  mean  and  standard  deviation  for  task 
duration  the  Pearson  Tukey  technique  is  appropriate  (Klastorin,  2011).  Both  the  standard 
deviation  and  the  mean  are  assumed  to  be  normally  distributed. 

Given  than  some  the  data  collected  was  from  a  subject  matter  expert  (SME), 
Klastorin  (2011)  suggested  that  experts  have  difficulty  estimating  with  great  accuracy. 
The  SME  at  DLA  DS  in  Camp  Pendleton  is  the  only  trained  verifier  who  has  the 
credential  to  access  databases  to  conduct  the  item  research.  Therefore,  the  verifier  is 
within  bounds  for  the  author  to  rely  on  his  expert  opinion.  However,  Klastorin  (2011) 
further  suggested  utilizing  the  Pearson  Tukey  technique  to  reduce  subjectivity  in  the 
SME’s  opinion.  The  Pearson  Tukey  formula  considers  error  in  the  estimation’s  mean  and 
variance  calculations.  The  resulting  formula  consider  these  factors,  as  shown  below: 

Mean  Task  Time  =  t5 o  +  0.185(t95  +  -2 150) 

Standard  deviation  =  (tg5  -  t5)/ 3.25 

where:  t5 0  =  median 

tg  5  =  95th  Percentile 
ts  =  5th  Percentile 

Traditionally,  the  Monte  Carlo  simulation  approach  computes  the  probability 
safety  lead-time  that  the  task  will  be  completed  on  time  (Klastorin,  2011).  Therefore,  in 
the  search  to  fully  estimate  a  task’s  time  completion,  this  study  used  the  Monte  Carlo 
technique.  Furthermore,  the  researcher  used  the  Crystal  Ball  add-on  application  to 
Microsoft  Excel  to  simulate  the  scenario  for  10,000  times.  Then,  a  lognormal  distribution 
parameter  was  used  in  order  to  get  positive  time  result  as  only  non-negative  values  are 
meaningful  in  this  case.  The  distribution  is  skewed  because  it  doesn’t  give  negative  date 
and  time  that  are  not  possible  occurrences  (GAO,  2009b).  Figure  8  presents  the 
distribution  parameter’s  description  and  application  justification. 
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Distribution 

Description  Shape 

Typical  application 

Lognormal 

A  continuous  distribution 
positively  skewed  with  a 
limitless  upper  bound  and 
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Probability 

fc/\ 
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know  how  to  scale  the  standard 
deviation,  which  is  needed  for  this 
distribution 

Values 

Normal 

Used  for  outcomes  likely  to 
occur  on  either  side  of  the 
average  value;  symmetric 
and  continuous,  allowing  for 
negative  costs  and  durations. 

In  a  normal  distribution,  about 

68%  of  the  values  fall  within  one 
standard  deviation  of  the  mean 

ProOaDillty  ^ 

V 

//  \ 

s  / 

To  assess  uncertainty  with  cost 
estimating  methods;  standard 
deviation  or  standard  error  of  the 
estimate  is  used  to  determine 
dispersion.  Since  data  must  be 
symmetrical,  it  is  not  as  useful 
for  defining  risk,  which  is  usually 
asymmetrical,  but  can  be  useful  for 
scaling  estimating  error 

Values 

Figure  8.  Probability  Distribution  Description  Shapes  and  Application 

(from  GAO,  2009b) 


Finally,  adjusting  the  data  for  uniformity  to  actual  cost  is  considered.  Given  that 
some  of  the  data  was  generated  in  2008  and  other  data  in  2013,  normalization  is  required. 
Similarly,  the  historical  dollar  value  data  must  be  normalized  for  inflation.  Therefore,  the 
researcher  used  the  military  construction  (MILCON)  inflation  factor  to  convert  the 
funding  profiles  and  determined  the  then  year  (TY)  into  fiscal  year  2014  (FY14)  to 
establish  a  baseline  of  cost.  Additionally,  the  default  setting  “Show  empty  data  as  zero” 
was  enabled  in  Excel  to  automatically  treat  blanks  cells  as  zero.  Appendix  B  showcases 
the  treatment  of  empty  cell  the  method.  Furthermore,  the  results  from  the  data 
normalization  were  used  to  estimate  the  time-task-duration  to  account  for  it  in  the  CBA 
section  of  the  thesis. 

The  following  disclaimer  should  be  noted:  Crystal  Ball  is  licensed  for  educational 
purpose  by  The  Naval  Postgraduate  School;  therefore,  the  Crystal  Ball  charts  contain  the 
statement  “Not  for  Commercial  Use.” 

In  summary,  normalizing  reduces  dispersion  of  data  points  and  maintains  constant 
and  accurate  information.  Therefore,  it  is  imperative  to  adapt  the  method  and  techniques 
described  in  this  section  in  order  to  be  applied  in  this  study. 
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c. 


TECHNOLOGY  READINESS  METHODOLOGY  STEPS 


Using  the  TRL  table  established  in  Chapter  II,  this  study  provides  results  of  the 
three  companies.  According  to  the  GAO  (2003),  the  TRL  tool  is  used  to  assess 
technology’s  maturity.  The  TRL  methodology  is  widely  known  in  the  DOD  for  its  best 
practice  approach  of  ranking  technology’s  maturity.  The  benefit  of  evaluating  three 
companies’  technology  provides  controls  to  reduce  the  risk  of  running  the  project  over 
schedule  and  over  budget.  The  foundation  to  make  the  technology  assessment  using  the 
TRL  tool  was  outlined  in  Chapter  II. 

In  this  study  three  commercial-off-the-shelf  object/shape  recognition  tools  were 
evaluated.  This  study  followed  the  TRA  skeletal  template  from  the  DOD  TRL  Guidance 
and  adopted  the  DOE  report  format.  The  COTS  with  the  highest  TRL  will  be 
recommended  for  further  maturation.  The  TRA  report  consists  of  the  following 
information: 

1 .  Overview  of  object/shape  recognition  technology  profde. 

2.  Technology’s  goal  with  regard  to  what  it  does  and  affects. 

3.  Benefits  with  regard  to  how  the  technology  benefits  businesses  that  use  it. 

4.  Critical  area(s)  of  challenge  such  as  finding  a  demonstration  and  any  risk 
implementing  it  at  DLA  DS. 

5.  Qualitative  summary  of  technology’s  technology  readiness  level  (TRL) 
using  the  TRL  table  in  Chapter  I. 

6.  Statement  of  relevancy  to  DLA  DS,  which  provides  a  conclusion  with 
supporting  evidence  or  rationale  and  dissenting  opinion  of  technical 
aspects  information  that  may  be  deemed  pertinent  to  support  the  decision¬ 
maker. 

1,  Data  Sources  for  Technology  Readiness  Level  Assessment 

The  researcher  visited  three  companies  during  the  months  of  August  to  collect 

data  to  assess  the  technology  maturity  of  three  products.  The  researcher  spent  one  half 

day  at  each  of  the  company.  The  researcher  observed  demonstrations  of  the  technology 

products  at  each  firm,  taking  detailed  notes  and  asking  clarifying  questions  to  develop  a 

deep  understanding  of  each  product’s  capabilities.  The  researcher  also  collected 
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presentations  of  the  technologies,  and  white  papers,  pamphlets  and  videos  describing  the 
technologies.  This  data  is  summarized  in  Table  2. 


Table  2.  Data  Collected  from  Companies 


Company 

Data 

Quantity 

COGNEX 

Pamphlet 

2 

COGNEX 

Video  from  YouTube 

5  minutes 

Imaginestics 

Pamphlet 

2 

Imaginestics 

Pictures 

10 

Imaginestics 

PowerPoint  (1) 

10  slides 

Imaginestics 

Video 

20  minutes 

RMD 

White  paper 

1 

Overcoming  Data  Challenges:  The  researcher  gathered  the  primary  data  and 
information  used  for  this  study  during  business  trips.  The  purpose  of  the  business  trips 
was  to  observe  and  assess  the  technology’s  maturity,  and  conduct  a  cost  evaluation.  The 
researcher  visited  three  potential  companies  of  interest  and  communicated  via  telephone 
and  email  with  three  other  companies.  Admittedly,  there  were  challenges  in  gathering 
actual  technology  costs  posed  by  the  companies’  leadership  due  to  proprietary  technology 
rights,  privacy  policies  and  reluctance  to  advertise  actual  cost  without  negotiating  a 
contract.  However,  one  company  did  provide  examples  of  costs  for  use  in  this  thesis. 
Finally,  the  researcher  believes  that  the  evidence  obtained  during  the  study  provided 
reasonable  basis  to  provided  reasonable  accurate  information  in  order  to  make  an 
informed  analysis  conclude  with  recommendations.  The  study  does  not  endorse  one 
specific  company  over  another.  The  intent  of  the  study  is  to  offer  DLA  DS  decision¬ 
makers  as  much  information  relating  to  what  the  commercial  sector  has  to  offer  on 
object/shape  recognition  technology  as  possible,  so  that  they  can  make  the  most  infonned 
decision  about  implementing  technology  to  their  property  identification  process. 

D.  COST  BENEFIT  ANALYSIS  METHODOLOGY  STEPS 

The  CBA  considers  which  alternative  provides  the  highest  positive  tangible  and 
intangible  realized  net  benefits.  The  CBA  is  an  analytical  tool  to  support  the  decision- 
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making  process  when  choosing  between  competing  alternatives.  A  sensitivity  analysis 
supports  the  CBA  by  changing  assumptions  to  understand  risk  factors  that  alternatives 
inherit.  Boardman  et  al.  (2011)  proposed  the  analysis  to  calculate  social  benefit  for  each 
alternative:  net  social  benefit  (NSB)  equals  social  benefit  (B)  minus  social  cost  (C). 


NSB  =  B  -  C 


According  to  Boardman  et  al.  (2011),  there  are  three  major  types  of  CBA  and  one 
comparative  class  of  CBA: 

1 .  Ex  ante  CBA:  conducted  prior  to  the  project;  useful  to  show  whether 
resources  should  be  allocated  for  a  particular  project. 

2.  Ex  post  CBA:  conducted  at  the  end  of  the  project;  provides  information 
about  the  particular  class  of  project. 

3.  In  medias  res  CBA:  conducted  during  the  project. 

4.  Comparative  CBA:  compares  the  ex  ante  predictions  to  ex  post  results  for 
the  same  project. 

This  research  study  used  the  ex  ante  CBA  to  provide  information  to  DLA 
decision-makers  prior  to  making  an  investment.  Notwithstanding,  there  are  nine  basic 
steps  in  conducting  a  CBA  (Boardman  et  al.,  2011,  p.  6),  which  will  be  applied  in  the 
methodology  chapter.  The  details  of  all  the  steps  for  CBA  are  detailed  in  the  sections  that 
follow: 

Step  1 :  Specify  the  set  of  alternative  projects. 

Step  2:  Decide  whose  benefits  and  costs  count  (standing). 

Step  3:  Catalog  the  impacts  and  select  measurement  indicators. 

Step  4:  Predict  the  impacts  quantitatively  over  the  life  of  the  project. 

Step  5:  Monetize  (attach  dollar  values  to)  all  impacts. 

Step  6:  Discount  benefits  and  costs  to  obtain  present  values. 

This  analysis  normalized  the  wages  data  for  inflation.  Therefore,  a  nominal 
discount  rate  is  used  to  discount  cost  and  benefit  over  the  10-year  horizon  time. 
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Furthermore,  in  order  to  account  for  the  time  value  money,  this  research  used  a  7% 
discount  rate.  According  to  the  Circular  A-94,  when  analyzing  proposed  investment  the 
7%  rate  is  ideal  for  calculating  the  return  on  investment  in  the  private  sector  (OMB, 
1992). 

Step  7:  Compute  the  net  present  value  of  each  alternative.  The  net  present  value 
(NPV)  equals  the  present  value  of  benefits  minus  the  present  value  of  costs: 

NPV  =  PV  (B)  -  PV(C) 

Choose  the  alternative  with  the  largest  NPV. 

Step  8:  Perform  sensitivity  analysis.  To  shield  from  uncertainty  of  the  predicted 
impact,  this  study  conducted  a  yearly  variance  sensitivity  analysis  using  the  5%  lower 
bound  and  90%  upper  bound  confidence  intervals. 

Step  9:  Make  a  recommendation.  The  results  from  the  NPV  calculation  with  the 
alternative  with  the  largest  NPV  will  be  the  criterion  for  recommendation.  This  will  also 
consider  the  sensitivity  analysis  to  make  recommendations. 

For  this  thesis,  the  status  quo  alternative  is  compared  with  the  option  to 
implement  new  object/shape  recognition  technology.  This  thesis  builds  upon  this 
generally  accepted  approach  to  fully  analyze  the  costs  and  benefits  associated  with 
implementing  object/shape  recognition  technology  at  DLA.  As  a  general  rule,  CBA  is 
created  for  a  comprehensive  analysis. 

1.  Assumptions,  and  Data  Sources  for  Cost-Benefit  Analysis 

The  following  data  and  assumptions  are  used  for  conducting  the  cost  benefit 
estimates. 


a.  Assumptions 

In  order  to  estimate  benefits  generated  by  the  implementation  of  an  object/shape 
recognition  technology,  this  thesis  uses  the  opportunity  cost  approach.  In  essence,  the 
benefits  associated  with  the  new  technology  are  estimated  as  savings  in  labor  costs  that 
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do  not  have  to  be  paid  out  any  longer  once  the  labor-intensive  status  quo  is  replaced. 
Although  DLA  DS  is  a  non-profit  organization,  the  researcher  is  able  to  use  the  market 
comparable  approach  to  calculate  the  return  on  investment.  While  DLA  DS  does  not  have 
an  overall  comparable  commercial  market  force  to  compare  revenues,  the  above 
assumption  allows  the  researcher  to  use  the  market  comparable  labor  rates  as  a  form  of 
estimating  revenues  (Housel  et  ah,  2007).  In  order  to  conduct  a  break  even  or  return  on 
investment  analysis,  Jegers  (2008)  suggested  calculating  earned  revenues  from  other 
sources  rather  than  sales.  Therefore,  there  seems  to  be  no  reason  to  use  the  current 
general  schedule  (GS)  wages  as  market  comparable  revenues.  The  researcher  calculated 
the  annual  wages  in  dollars  per  hours  in  order  to  monetize  the  LSN  research  task,  the 
assumption  being  that  automating  the  identification  process  with  technology  will  save 
money  in  labor  cost  and  increase  productivity.  Additionally,  implementing  object/shape 
recognition  technology  will  not  replace  the  bar  code  readers  because  implementing 
object/shape  technology  is  for  the  identifying  items  without  information.  Furthermore, 
the  scope  of  this  thesis  limits  the  amount  of  variables  one  can  research  and  utilize  in 
accounting  for  how  the  technology  will  generate  more  market  comparable  revenues. 

b.  Data  on  Employee  Wages  and  Number  of  Disposal  Operations 
Employees 

The  researcher  drew  from  the  GS  pay  chart  to  indicate  the  average  labor  rates  for 
employees  who  conduct  disposal  services  at  DLA  DS,  Camp  Pendleton,  California.  The 
cost  of  labor  is  used  to  conduct  the  NPV  to  satisfy  a  market  comparable  approach  in 
tenns  to  calculate  the  return  on  investment.  The  GS  pay  table  is  illustrated  in  Figure  9. 
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SCHEDULE  1- -GENERAL  SCHEDULE 


(Effective  on  the  first  day  of  the  first  applicable  pay  period  beginning  on  or  after  January  1,  2014) 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

GS-1 

317,981 

318, 582 

319,180 

319,775 

320,373 

320,724 

321,315 

321,911 

321,934 

322,494 

GS-2 

20,217 

20,698 

21,367 

21,934 

22,179 

22,831 

23,483 

24,135 

24,787 

25,439 

GS-3 

22,058 

22,793 

23,528 

24,263 

24,998 

25,733 

26,468 

27,203 

27,938 

28,673 

GS-4 

24,763 

25,588 

26,413 

27,238 

28,063 

28,888 

29,713 

30,538 

31,363 

32,188 

GS-5 

27,705 

28,629 

29,553 

30,477 

31,401 

32,325 

33,249 

34.173 

35,097 

36,021 

GS-6 

30,883 

31,912 

32, 941 

33,970 

34, 999 

36,028 

37,057 

38,086 

39,115 

40,144 

GS-7 

34,319 

35,463 

36, 607 

37,751 

38,895 

40,039 

41,183 

42,327 

43,471 

44,615 

GS-8 

38,007 

39,274 

40,541 

41,808 

43,075 

44,342 

45,609 

46,876 

48,143 

49,410 

GS-9 

41,979 

43,378 

44,777 

46,176 

47,575 

48,974 

50,373 

51,772 

53,171 

54,570 

GS-10 

46,229 

47,770 

49,311 

50,852 

52,393 

53,934 

55,475 

57,016 

58,557 

60,098 

GS-11 

50,790 

52,483 

54, 176 

55,869 

57,562 

59,255 

60,948 

62,641 

64,334 

66,027 

GS-12 

60,877 

62,906 

64,935 

66,964 

68,993 

71,022 

73,051 

75,080 

77,109 

79,138 

GS-13 

72,391 

74,804 

77,217 

79,630 

82,043 

84,456 

86,869 

89,282 

91,695 

94,108 

GS-14 

85,544 

88,395 

91,246 

94,097 

96, 948 

99,799 

102,650 

105,501 

108,352 

111,203 

GS-15 

100,624 

103,978 

107,332 

110,686 

114,040 

117,394 

120,748 

124,102 

127,456 

130,810 

Figure  9.  General  Schedule  Pay  Table  for  2014 
(from  Executive  Order  No.  13,655,  2013) 


The  DLA  DS  at  Camp  Pendleton  structure  was  collected  to  identify  how  many 
employees  are  part  of  the  disposal  operations.  During  the  study,  two  positions  were 
vacant  as  illustrated  in  Table  3.  The  two  employee  functions  include  the  Disposition 
Services  Representative  (DSR)  supervisor  or  a  DSR,  the  pay  plan  is  under  the  GS  rules  in 
accordance  to  the  series  standard  1104  classifying  the  employee  as  a  Property  Disposal 
Clerical  or  Technician  pay  grade  hired  under  the  available  positions.)  In  this  case,  the 
grades  are  GS  9  through  GS12.  Figure  9  identifies  wages  for  every  respective  position. 
For  this  study,  the  GS  1 1  is  step  6,  whose  salary  is  $59,255,  consistent  with  the  pay  scale 
the  verifier  at  Camp  Pendleton  holds. 


Table  3.  DLA  DS  at  Camp  Pendleton,  California,  Disposal  Operations 
Structure  as  of  July  2014  (from  DLA  DS,  2014) 


Disposal 

Operations  Si 

tructure 

Vacant/Full 

Function 

Pay  Plan 

Series 

Grade 

Vacant 

DSR 

Supervisor 

GS 

1104 

12 

Full 

DSR 

GS 

1104 

11 

Full 

DSR 

GS 

1104 

11 

Full 

DSR 

GS 

1104 

09 

Vacant 

DSR 

GS 

1104 

09 

Full 

DSR 

GS 

1104 

09 

Full 

DSR 

GS 

1104 

09 

Full 

DSR 

GS 

1104 

11 
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E.  SUMMARY 

This  chapter  outlined  the  foundational  method  for  conducting  the  CBA  in  order  to 
make  infonned  recommendations.  This  chapter  described  the  steps  taken  to  analyze  the 
item  processing;  including  the  method  used  to  normalize  the  data  to  ensure  unifonnity 
and  integrity.  The  chapter  also  presented  the  assumptions  and  data  sources  for  the  cost 
benefit  analysis.  The  researcher  took  into  account  that  collecting  data  from  a  SME  has 
biases  such  that  to  reduce  their  opinion’s  subjectivity  the  Person  Tukey  is  an  appropriate 
formula  to  consider  for  error.  The  next  chapter  will  present  the  analysis  of  DLA  DS’s 
purpose  and  operational  method,  and  discuss  the  assessment  results  on  COTS  of 
objects/shapes  recognition  technology. 
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IV.  ANALYSIS  OF  THE  CURRENT  PROPERTY  PROCESS  AT 
DEFENSE  LOGISTICS  AGENCY  DISPOSITION  SERVICES 
AND  MATURITY  ASSESSMENT  OF  OBJECT/SHAPE 
RECOGNITION  TECHNOLOGY 


In  support  of  the  DLA  mission  and  vision,  DLA  Disposition  Services 
supports  the  Warfighter  and  protects  the  public  by  providing  worldwide 
reverse  logistics  solutions.  Our  goal  is  to  become  the  Department  of 
Defense’s  Reverse  Logistics  Center  of  Excellence 

— DLA  Disposition  Services  Transformation  Plan  2014-2019 

A.  OVERVIEW 

Chapter  III  explained  the  method  by  which  this  thesis  explored  the  organization 
process  with  Defense  Logistics  Agency  Disposition  Services  (DLA  DS).  This  chapter 
presents  the  background  on  DLA  DS  management’s  concerns  to  contextualize  this  study, 
and  presents  and  discusses  the  results  of  that  analysis  of  the  current  property  process  at 
DLA  DS.  In  addition,  this  chapter  provides  an  overview  of  DLA  DS’  purpose, 
operational  process  activities,  and  managements’  concerns  and  details  the  results  from  the 
assessment  on  objects  shapes  recognition  technology  prototypes  from  three  companies. 

1.  Introduction 

On  September  1972,  the  Disposition  Services  (DS)  was  established  as  a  division 
of  the  Defense  Logistics  Agency  (Grasso,  2014).  The  mission  of  the  DS  is  the 
management  of  the  DOD’s  assets  that  have  reached  the  end  of  their  life  cycle. 
Specifically,  DS  focuses  on  solutions  to  dispose  property  no  longer  wanted  from  all  the 
Marines,  Army,  Navy,  and  Air  Force,  and  other  federal  agencies.  Today,  the  DLA  DS 
management  is  interested  in  improving  the  efficiency  of  the  item  processes.  One  potential 
way  to  improve  efficiency  of  the  DLA  DS  processes  is  the  use  of  object/shape 
recognition  technology,  identifying  as  a  way  to  reduced  costs  by  DLA  DS  (2008)  study. 
This  section  describes  the  DS  purpose  and  motivation  for  considering  technology  to 
achieve  DLA  DS  goals  and  underlines  the  property  identification  process,  inefficiency, 
problems  with  the  processing  system  that  creates  backlog,  and  analysis  to  resolve  the 
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backlog  issue.  To  illustrate  the  points,  observations  made  by  the  researcher  during  the 
visit  at  the  DLA  Disposition  Services  at  Camp  Pendleton,  California,  will  be  provided. 

2.  Defense  Logistics  Agency  Disposition  Services  Activity 

a.  Disposition  Services  Purpose 

The  DS  provides  support  to  the  DOD  in  109  different  locations  worldwide  that 
including  15  countries  and  41  states,  employing  over  1,500  employees.  DS  Headquarters 
is  in  Battle  Creek,  Michigan,  its  location  meant  to  control  and  support  its  mission  (DLA 
DS,  2013). 

The  broad  purpose  of  a  DS  network  is  to  facilitate  the  DOD  in  property  disposal 
around  the  world.  DS  calls  the  services  “reverse  logistics.”  Reverse  logistics  is  defined  as 
the  operations  that  accept  property  from  the  branches  of  services  to  screen  it  for 
redistribution  (DLA  DS,  2013).  To  manage  accountability  and  control  around  the  world, 
DS’s  purpose  statement  says  the  following: 

Our  purpose  is  to  serve  as  the  Reverse  Logistics  provider  for  the 
Department  of  Defense  to  include  services  for  the  disposal  of  material  no 
longer  needed  for  national  defense,  comply  with  legislative  and  regulatory 
requirements,  protect  the  public  from  dangerous  defense  items,  and  pursue 
maximum  value  for  tax  dollars.  Our  Reverse  Logistics  capacity  includes 
responsibility  for  property  reuse  (including  resale),  hazardous  property 
disposal,  demilitarization,  precious  metals  recovery  and  recycling  program 
support,  worldwide.  Reverse  Logistics  is  more  than  property  disposal;  it  is 
the  process  of  planning,  implementing,  and  controlling  the  efficient,  cost 
effective  flow  of  excess  materials,  in-process  inventory  and  related 
information  from  the  point  of  declaration  of  excess  for  the  purpose  of 
recapturing  value  or  proper  disposal.  (DLA  DS,  2014a,  p.  1) 

Reverse  logistics  includes  the  following  disposal  choices:  reuse;  transfer  property 
within  inter  branch  of  service;  donate  property  to  state,  local  or  non-profit  organization 
and  sell  it  to  the  public;  or  destroy  it  to  turn  it  into  scrap  to  prevent  further  use. 
Consequently,  DS  goals  are  aligned  with  the  DLA  to  accomplish  the  reverse  logistics 
purpose. 
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b.  Motivation  for  Considering  Technology 

As  important  as  DS  is  to  the  DLA,  the  DLA  director  requires  that  all  divisions 
“Improve  process  and  productivity”  in  order  to  become  responsible  stewards  of 
taxpayer’s  dollar  (DLA,  2014).  Additionally,  DS  has  the  added  pressure  of  fiscal 
obligations  and  budgetary  cuts  such  that  DS  continuously  pursues  decreasing  operations 
cost.  Therefore,  goal  achievement  and  sustainment  is  of  DS  top  priority,  so  factors  that 
affect  operation  cost  are  continuously  examined. 

To  accomplish  reductions  of  operations  cost,  DS  conducts  studies  to  identify  any 
cost-reducing  opportunities.  In  2008  (DLA  DS,  2008),  a  study  to  change  the  20-step 
property  acceptance  process  was  concluded  at  one  of  DS’s  facilities  located  in  Stockton, 
California.  The  study  resulted  in  streamlining  the  process  to  seven  steps  using  the  Lean 
Six  Sigma  logic.  The  change  in  process  resulted  in  time  savings  equivalent  to  40%  (DLA 
DS,  2008).  The  study  concluded  with  a  recommendation  for  a  future  study  to  exploit 
wireless  capabilities  (DLA  DS,  2008). 

This  thesis  aims  to  analyze  DS  item  process,  conduct  a  technology  readiness 
assessment  and  conduct  a  cost  benefit  analysis  in  effort  to  finding  a  solution  to  improve 
the  process  with  technology.  To  achieve  this  goal,  this  thesis  adopted  two  DS  strategic 
objectives  set  by  the  DLA  DS  Director  Transformational  Plan  2014 — 2019: 

1 .  Embrace  innovation  and  leverage  industry:  “Employ  industry  and 
academia  to  pursue  technological  advancement  in  reverse  logistics 
solutions”  (DLA  DS,  2014a,  p.  10). 

2.  Identify  opportunities  for  process  enhancement  to  reduce  cost:  “Optimize 
Network  to  provide  accountable,  compliant,  and  agile  property  handling 
(internal  operating  system  strategy)”  (DLA  DS,  2014a,  p.  10). 

3.  Property  Disposal  Process 

In  2008,  efforts  to  improve  the  property  process  resulted  in  a  streamlined  process 
that  saved  time  through  the  lean  approach.  Currently,  DS  continues  to  process  property 
using  the  Lean  Six  Sigma  standards.  However,  DS  recognizes  that  the  improved  seven- 
step  process  can  be  tweaked;  therefore,  DS  is  interested  in  additional  improvements. 
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Hence,  in  order  to  seek  improvements,  one  needs  to  understand  the  process  flow  to 
identify  inefficiencies,  drawbacks,  and  problems. 

DS  manages  property  for  disposal  and  the  choices  include  one  of  the  following: 
reutilization,  transfer,  donation,  resale  and  ultimate  disposal  as  scrap.  In  this  context,  the 
holistic  process  is  depicted  in  Figure  10. 
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Figure  10.  Disposition  Services  Process  Disposal  Overview  (from  DLA  DS,  n.d.) 


Figure  10,  presents  the  overall  flow  of  property  acceptance  conducted  at  DS.  The 
items  are  received  from  all  four  branches  of  service — Army,  Navy,  Marine  Corps  and  Air 
Force — and  other  federal  agencies.  Then  the  items  are  individually  screened  for  proper 
disposal  decision.  Some  of  these  items  are  redistributed  for  reutilization  by  any  branch  of 
service  or  transferred  to  federal  agencies.  Major  recipients  of  donated  items  go  to  state 
and  local  agencies,  like  the  fire  department  or  police  stations.  Additionally,  some  items 
that  are  safe  for  public  use  are  sold  through  public  auctions  and  scraped  items  are  sold 
through  business  partners.  Finally,  items  that  cannot  be  sold  to  the  public  due  to  potential 
hazard  or  to  prevent  putting  them  into  the  hands  of  an  adversary  are  mutilated  or  stored  in 
military  facilities. 

The  property  disposal  process  begins  when  the  customer  completes  the  mandatory 
disposal  turn-in  document  (DTID),  DD  Form  1348  1A,  and  associated  certificates 
depending  on  the  type  of  property  (further  details  about  how  the  document  is  generated  is 
emphasized  later  in  the  section).  The  property  is  accepted  from  the  customer  as  a 
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conveyance  and  tagged  with  a  receipt  control  number  (RCN).  The  RCN  identities  the 
customer,  tracks  the  time  it  takes  to  process  the  items,  reduces  dual  effort,  organizes  for 
precedence  processes,  order,  and  optimizes  limited  resources.  Conveyance  is  then  tracked 
and  identified  as  an  RCN. 

Turned-in  property  at  DS  arrive  in  pallets,  triwalls  full  of  line  items  (Lis)  or  single 
LI.  DS  employees  are  hired  in  any  position  under  the  Deposition  Service  Representative 
(DSR)  and  are  known  and  referred  to  full-time  employees  (FTEs).  FTEs  store  the  RCN 
property  that  cannot  be  inspected  for  disposal  decision  upon  arrival,  and  at  that  moment  it 
becomes  backlog  for  future  process  in  precedence  of  arrival.  Items  that  are  inspected  on  a 
daily  basis  are  broken  down  into  individual  Lis,  and  then  the  usable  property  is  retained 
for  42  days  of  screening.  During  the  screening  phase,  the  retained  property  is  marketed 
for  reutilization,  transfer,  or  donation  (RTD)  to  the  branches  of  services,  or  local  or 
federal  agencies,  as  noted  in  Figure  1.  After  42  days,  the  LI  that  is  not  RTD  is  handled 
again  and  turned  over  to  government  liquidations  contractors  to  sale  to  the  public. 
Property  that  requires  demilitarization  (DEMIL)  is  only  available  for  a  14-day  period  per 
the  DOD  screening  cycle  and  then  sent  to  a  DEMIL  site  for  proper  disposal  and 
destruction.  DEMIL  property  is  not  made  available  for  public  sale.  Figure  14  illustrates  a 
timeline  of  the  process. 


Figure  11.  Disposition  Services  Process  Disposal  Timeline  (from  DLA  DS,  n.d.) 


Figure  1 1  displays  the  days  required  to  accomplish  property  disposal.  The  RTD 
activities  occur  between  days  1  and  42.  In  the  incubation  period,  days  1-7,  LI  is  pending 
screening  to  decide  appropriate  disposal.  Annually,  on  average,  RCNs  wait  7.6  days  to  be 
processed  for  screened  for  demilitarization  or  downgraded  for  scrap.  During  the 
reutilization-screening  period,  1-14  days,  Lis  are  screened  for  the  RTD  program.  During 
the  transfer-screening  period,  the  LI  is  initially  requested  to  be  RTD.  During  the 
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donation  period,  36-40  days,  Lis  are  released  under  the  RTD  program  for  reuse.  These 
items  can  then  be  transferred  to  the  federal  agency  that  needs  it.  The  law  authorizes  these 
transfers  to  federal  agencies,  state  agencies,  or  police  stations.  Additionally,  items  can  be 
donated  to  non-governmental  agencies  or  humanitarian  aid  organizations.  Annually  DS 
takes,  on  average  32  days  to  ship  out  under  a  document  identifier  form  Materiel  Release 
Order  (MRO).  Between  40  and  42  days,  last  chance,  are  the  last  days  items  can  be 
requested  for  RTD  by  any  branch  of  service,  state  agency,  or  local  agency.  Finally, 
during  ultimate  disposal  (UD),  the  end  of  the  cycle,  some  property  can  be  sold  to  the 
public  when  it  is  deemed  safe  for  sale,  destroyed,  or  transferred  to  a  long-term  storage 
facility.  The  flow  diagram  (see  Figure  12)  provides  amplifying  explanation  that  occurs  at 
every  step  of  the  disposal  process. 
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DISPOSITION  FLOW  DIAGRAM 
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Figure  12.  Process  Flow  Diagram  of  Items  Turned  in  for  Proper  Disposal 

(from  DLA  DS,  2008) 


All  items  turned  in  for  disposal  at  DS  go  through  these  internal  sequence  of 
events:  (1)  receive  items  received  into  conveyance  and  issued  RCN  to  store  in  backlog 
storage,  (2)  conveyance  broken  down,  (3)  screening,  (4)  item  sorting,  (5)  unidentified 
item  research  is  conducted,  (6)  code  for  disposition  identified,  (7)  boxed  in  tri-wall 
containers  for  final  life  cycle  procedure.  The  process  depicted  in  Figure  12  provides  the 
steps  the  Lis  undergo.  All  items  turned  in  undergo  through  this  cycle: 

1 .  Receive  items:  Conveyance  is  tagged,  but  it  may  have  up  to  50 
individual  line  items.  Approximately  70%  of  all  Lis  received  are 
downgraded  upon  receipt  for  sale  as  scrap  by  weight  (DLA  DS, 

2014a). 
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2.  Breakdown  the  conveyance:  Batches  are  itemized  into  individual  Lis 
for  inspection.  This  accounts  for  the  remaining  30%  of  Lis.  These  Lis 
end  up  in  the  RTD  pipeline. 

3.  Screen:  As  noted  above,  the  RTD  screening  process  includes  a  “42- 
day  staggered  availability  period  that  offers  property  for  reuse  first  to 
DOD  consumers;  next  other  federal  government  consumers  and  last, 
the  property  is  available  for  donation  to  authorized  nonprofit 
organizations”  (DLA  DS,  2014a).  After  42  days,  the  items  are  sent  to 
the  Control  Property  Center  (CPC)  for  sorting.  Figure  1 1  presents  the 
screening  timeline. 

4.  Sorting:  Batches  are  sorted,  identified  for  restrictions  to  sell  it  to  the 
public  use  demilitarized  (scrap)  or  long  tenn  storage. 

5.  Managed  unidentified  item:  The  FTE  verifies  and  conducts  research 
using  one  or  all  three  databases  (i.e.,  Federal  Logistics  Data  and  FLIS 
=  Federal  Logistics  [FEDLOG],  WebFLIS  and  Haystack). 

6.  Code  for  disposition:  The  items  that  cannot  be  verified  are 
downgraded  to  a  scrap  category. 

7.  Managed  boxed-in-tri-wall  containers:  Items  with  codes:  demilitarized 
or  mutilate  (MUT)  for  total  destruction  to  the  point  of  scrap  are  boxed 
and  sent  to  the  nearest  Centralized  DEMIL  Division  (CDD). 

After  42  days,  in  the  last  three  steps  of  the  process,  the  FTEs  screen  items  for 
ultimate  disposal.  Steps  5  and  6  illustrated  in  Figure  3  are  the  focus  of  this  project.  Step  5 
is  a  crucial  step  because  time,  resources  and  money  are  consumed  because  the  FTE 
conducts  research.  In  Step  6,  the  FTE  decides  to  scrap  unidentifiable  Lis.  Conversely, 
when  LI  information  is  discovered,  the  FTE  detennines  to  give  the  LI  one  more 
opportunity  to  RTD. 

In  all,  the  items  are  disposed  through  RTD,  resale,  scrap,  or  long-term  storage. 
There  is  no  shortage  of  available  Lis  as  items  continuously  arrive.  Similarly,  at  the  other 
end  of  the  process  items  are  disposed  but  sometimes  with  challenges.  Where  does  the 
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disposed  property  end  up?  According  to  DLA  DS  (2014a)  these  items  have  a  final 
destination  and  the  percentages  are  explained: 

Of  the  total  items  that  enter  the  disposition  process,  approximately  85%  of 
items  are  disposed  as  scrap  or  Hazardous  Waste  (HAZMAT),  6%  of  items 
are  claimed  during  the  RTD  screening  cycle  and  9%  of  the  items  are 
eventually  sent  to  sales  when  not  requested  for  reutilization.  This  type  of 
property  includes  Hazardous  Material  (HAZMAT),  Hazardous  Waste  and 
items  with  specific  DEMIL  requirements,  all  of  which  are  handled  under 
specialized  rules.  A  fraction  of  the  items  that  are  identified  as  DEMIL 
coded  “B”  or  “Q,”  (due  it  being  hazardous  to  release  to  public),  are  sent  to 
Long  Term  Storage.  (DLA  DS,  2014a,  p.  7) 

In  summary,  in  2008,  the  efforts  to  improve  received  property  resulted  in  a 
streamlined  process  that  saved  time.  Currently,  DS  continues  to  process  property  using 
the  Lean  Six  Sigma  standards.  DS  recognizes  the  improved  7-step  process  can  be 
tweaked;  therefore  DS  is  interested  in  additional  improvements.  The  next  section 
provides  observations  that  aim  to  identify  any  inefficiencies,  challenges,  drawbacks  and 
problems  with  the  property  process. 

a.  Inefficiencies 

From  a  qualitative  perspective,  the  current  process  requires  multiple 
“touches”  of  the  property  including  handling,  segregating,  processing,  and 
shipping.  This  relatively  “high  touch”  and  risk  adverse  environment  is 
inefficient,  costly  and  burdensome  to  our  customers  and  ourselves  and 
must  be  reduced. 


— DLA  DS,  2014a,  p.  7 

The  previous  sections  described  the  disposition  process  and  the  goals  to  cost 
savings  associated  with  improving  the  item  disposition  process.  However,  some  of  these 
efforts  have  been  challenged  with  the  ongoing  inefficiency  to  effectively  identify  Lis. 
Examining  the  operational  flow  in  Figure  3  reveals  where  potential  challenges  and 
“bottlenecks”  are  created.  The  results  of  the  analysis,  it  turned  out,  show  that  researching 
items  is  not  the  only  area  of  concern  with  regard  to  process  inefficiency.  About  less  than 
1%  of  accepted  items  at  DS  facility  in  Camp  Pendleton  are  unlabeled  which  creates  for 
an  additional  time  to  research  the  missing  infonnation.  Actions  to  dispose  of  Lis  may  be 

slow  due  to  the  inaccuracy  of  information  recorded  in  the  DTID,  accepted  LI  is  obsolete 
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and  risk  adverse  culture  creates  drawbacks  and  impacts  progress.  Additionally,  a  greater 
number  of  accepted  items  may  be  more  than  the  amount  of  items  that  are  processed  on  a 
daily  basis.  This  inefficiency  is  precisely  quoted  above  and  cited  from  the  DLA  DS’ 
Transformation  Plan  2014-2019  (DLA  DS,  2014a). 

In  general,  items  that  are  properly  marked  occur  when  the  disposal  request,  DTID, 
is  generated  via  the  Dispositions  Services’  website  or  the  Distribution  Standard  System 
(DSS),  which  comes  from  Federal  Logistics  (FLIS).  DSS  is  the  automated  property 
accounting  and  inventory  management  system  used  by  DS  to  process  personal  property 
through  the  disposal  cycle  (DLA  DS,  2013c,  p.  1).  When  inputting  a  DTID  with  a  valid 
NSN  into  DSS,  the  system  automatically  populates  the  corresponding  DEMIL  code  that 
has  been  assigned  in  FLIS.  If  the  screen  shows  a  DEMIL  code  that  is  different  from  what 
is  believe  the  code  should  be,  then  the  customer  can  make  corrections  to  enter  the  proper 
information  and  that  alert  allows  for  verification  to  correct  for  coding  errors  so  that  the 
proper  item  is  submitted  for  disposal.  According  to  the  DLA  DS  training  manual,  there 
are  benefits  for  filling  the  DD  1348  through  the  web-based  Electronic  Disposal  Turn-in 
Document  (ETID)  program: 

ETID  allows  the  customer  to  electronically  submit  turn-in  documentation 
to  the  disposition  services  site.  This  program  will  pre -populate  many  of 
the  fields  for  NSN  items,  nomenclature,  DEMIL  code,  unit  price,  etc.  It 
includes  drop  down  menus  for  other  fields  for  quick  reference.  In  addition 
to  electronically  preparing  your  turn-in  documentation,  the  program 
allows  printing  a  completed  DD  1348-1A,  shipping  paper,  required 
DEMIL  certifications,  and  bar  codes  the  DD  1348-1  A.  One  of  the  benefits 
of  using  ETID  is  that  it  allows  the  disposition  sites  to  review  the  turn  in 
paperwork  prior  to  physical  movement  of  the  item  and  any  questions  can 
be  quickly  resolved.  ETID  requires  an  account  to  be  established  with  user 
id  and  password.  (DLA  DS,  2014b,  p.  20) 

In  contrast,  there  are  some  anomalies  in  the  documentation’s  accuracy  when  is 
handwritten.  The  DLA  DS  website  is  able  to  properly  correct  for  errors,  whereas 
handwritten  documents  prepared  by  the  customer  at  the  generating  activity  may  not  have 
all  or  accurate  information.  Nonetheless,  DD  1348s  are  manually  corrected,  as  noted  in 
Figure  15,  but  the  progress  is  sluggish  when  items  without  documentation  or  marking 
tags,  or  that  are  obsolete,  are  accepted  in  the  facility.  According  to  DLA  DS  standard 
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operation  procedure  (SOP)  “Disposition  Service  Customers  may  turn  property  in 
manually  or  electronically  via  ETID.  Customers  who  manually  prepare  the  DD  Form 
1348-1A  should  be  encouraged  to  use  ETID.  ETID  will  simplify  and  reduce  Generator 
time  processing  items  for  tum-in”  (DLA  DS,  2013c,  p.  4). 

Figure  13  illustrates  two  different  scenarios:  on  the  left,  items  are  properly 
marked  and  DD  1348s  are  enclosed  in  the  packing  protector;  and  items  on  the  right  side 
of  the  figure  are  items  on  a  pallet  without  DD  1348  forms. 


Figure  13.  Properly  Labeled  Items  and  Unlabeled  Items 
Note.  The  pictures  were  taken  by  the  researcher  in  DLA  DS  at  Camp 
Pendleton,  California,  in  July  2014 


DS’  item  process  faces  efficiency  challenges  and  an  additional  FTE  position — 
Verifier,  is  established.  The  verifier  researches  information  in  order  to  identify  unmarked 
property.  Some  of  the  challenges  posed  are  rapidly  finding  information  for  obsolete  items 
due  to  their  arrival  at  DS  without  manufacture  information  and  part  number.  The 
difficulty  with  obsolete  items  is  that  electronic  information  is  not  readily  available. 
Proper  identification  consumes  the  verifier’s  time  in  searching  for  the  information  on  the 
Internet  or  various  databases.  For  example,  the  verifier  at  DS  at  Camp  Pendleton 
conducts  research  on  the  Internet  for  items  that  do  not  have  paperwork  or  may  have  only 
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a  local  stock  number  (LSN).  An  LSN  stock  is  an  identification  code  generated  by  the 
owning  unit  prior  to  being  turned  in  to  DS.  However,  the  code  is  not  linked  to  the  actual 
part  number,  manufacturer  or  national  stock  number  (NSN)  information,  because  the 
owning  unit  may  not  have  had  that  infonnation  in  the  first  place.  This  disparity  of 
information  leads  to  adding  additional  steps  in  the  disposition  process  that  can  result  in 
additional  time  to  the  process.  On  average,  the  verifier  spend  about  13  minutes  to 
research  an  item’s  information.  Figure  14  shows  the  results  for  task  mean  time 
calculation. 


10.000  Tnats 


Frequency  View 


9.782  Displayed 


Figure  14.  Computer  Generated  Task  Time  using  Expert  Opinion  and 

Pearson  Tukey  Formula 


Figure  14  shows  a  simulation  of  the  task  time,  using  a  Monte  Carlo  simulation  on 
the  Crystal  Ball  (CB)  application.  The  estimate  given  by  the  DS  specialist  verifier  was  a 
time  requirement  of  5-10  minutes  to  finish  the  task.  However,  the  verifier  claims  to  have 
taken  him  up  30  minutes.  By  using  the  Pearson  Tukey  formula  to  take  into  account  any 
expert  error,  and  running  the  Monte  Carlo  simulation  on  CB,  the  findings  show  that  the 
chance  of  finishing  the  task  in  5-10  minutes  is  about  8-43%,  respectively,  and  the  chance 
of  finishing  the  task  in  30  minutes  is  about  3.5%.  However,  CB  shows  a  61%  chance  of 
finishing  the  task  in  13  minutes.  Therefore,  the  13-minute  mean  time  is  used  for  this 
analysis.  The  lognormal  curve  gives  the  probability  of  finishing  the  task  at  any  particular 
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time.  According  to  the  Oracle®  Crystal  Ball’s  guidebook  (Oracle®  Crystal  Ball,  Fusion 
Edition,  [Version  11.1.1.1.00],  n.d.),  the  results  in  the  diagram  are  interpreted  as  follow: 
the  forecast  chart  (the  x-axis  value  that  is  the  mean)  frequency  of  about  280,  meaning  that 
the  interval  expressed  by  that  column  contains  280  values.  The  mean  has  a  probability  of 
less  than  0.03  (or  3%),  meaning  that  there  is  a  3%  chance  of  a  value  falling  within  this 
interval.  The  certainty  range  includes  all  values  between  -Infinity  and  +Infinity.  The 
certainty  level  is  100%.  The  display  range  excludes  only  one  trial  out  of  the  total  10,000. 

The  DS  at  Camp  Pendleton  has  currently  one  verifier  who  conducts  the  research 
of  identifying  items.  However,  management  acknowledges  that  utilizing  an  additional 
employee  could  alleviate  the  overwhelmed  single  verifier.  One  verifier  is  a  single  point  of 
failure.  For  example,  in  the  absence  of  the  verifier  no  one  can  pick  up  and  continue  to 
conduct  the  research  because  the  other  employees  are  not  trained  and  do  not  have  access 
to  the  password-enabled  databases.  Therefore,  the  verifier  is  the  only  one  who  has  been 
granted  access  to  the  commercial  website,  Haystack.  This  website  is  a  contracted  web 
search  engine  for  a  limited  number  of  users  across  DS.  Nonetheless,  researching 
unidentified  items  for  proper  decision  disposal  is  a  way  to  reduce  a  risk.  Safety  is  always 
considered  when  getting  rid  of  items  so  militarized  items  do  not  end  up  in  the  enemy’s 
hands  or  exposing  hazard  to  the  public.  DLA’s  core  mission  is  to  “protect  the  public.” 
Figure  15  displays  the  Haystack  on  a  computer  screen.  The  verifier  finds  and  writes  the 
disposal  information  that  was  missing  on  the  DD  1348  using  Haystack.  The  verifier 
claims  to  spend  an  average  time  of  10-15  and  up  to  30  minutes  researching  items 
consuming  a  half  day’s  work.  So,  for  the  purpose  of  this  research  the  13-minute  task  time 
per  item  is  utilized,  consistent  with  the  computer-generated  forecast  model. 
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Figure  15.  Filled  Disposal  Tumed-in  Document  with  Local  Stock  Number 
Information  and  Reached  on  Haystack  website 
Note.  The  picture  was  taken  by  the  researcher  in  DLA  DS  at  Camp 
Pendleton,  California,  in  July  2014 


Figure  15  displays  a  DD  1348  with  the  printed  LSN,  but  the  handwritten 
information  was  researched  by  the  verifier  in  Haystack.  Figure  15  illustrates  the  paper 
DTID,  with  the  item’s  LSN  typed  the  information  was  researched  in  the  Haystack  web- 
based  contracted  program. 

Due  to  their  nature,  disposed  military  items  that  may  pose  some  risk  to  the  public. 
The  effect  is  that  it  has  cultivated  a  risk  adverse  culture,  and  management  would  much 
rather  invest  time  identifying  items  rather  than  selling  them  as  is.  According  to  the 
Receiving  SOP  (2014c),  items  are  “challenged,”  per  se,  for  further  investigation  and 
verifiers  utilize  all  available  resources.  For  example,  the  verifier  encounters  the  following 
situation  and  follows  this  procedure: 

When  to  challenge — the  first  test  of  whether  the  DEMIL  code  of  an  item 
should  be  challenged  is  made  by  comparing  the  DEMIL  code  on  the  DTID 
[DD  Form  1348]  for  the  item  and  the  item  itself  to  the  DEMIL  code  in 
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Distribution  Standard  System  (DSS),  which  comes  from  FLIS.  If  these  are 
not  in  agreement,  the  DEMIL  code  may  need  to  be  challenged.  Research 
must  be  performed  to  determine  what  the  correct  code  should  be  according 
to  the  established  guidelines  in  the  DEMIL  Manual.  (DLA  DS,  2014c,  p.7) 

Selling  an  unidentified  item  without  disposition  instructions  could  potentially  be  unsafe 
to  the  public.  Furthermore,  this  process  ensures  that  items  do  not  end  up  being  reused  as 
weapons  by  adversaries.  Therefore,  management  ensures  high  effort  is  spent  researching 
the  items  (Inspector  General  of  the  Department  of  Defense,  2013). 

With  a  risk  adverse-culture,  employees  exhaust  their  means  to  ensure  that  they 
have  disposition  codes.  However,  this  creates  a  drawback  regarding  time  allocated  to  one 
item.  Excess  time  spent  on  one  item  can  be  a  contributing  factor  to  creating  a  backlog. 
Another  drawback  is  that,  within  the  levels  of  security  measures,  items  end  up  in  the 
scrap  pile  for  disposal  and  miss  an  opportunity  for  resale.  These  measures  are  taken  for 
safety  reasons.  Nevertheless,  slow  and  inaccurate  identification  reduces  timeliness  of  an 
item’s  disposal  and  the  opportunity  to  continuously  process  more  items.  The  ratio  of  Lis 
researched  in  a  week  compared  to  items  received  is  calculated  by  the  following  fonnula: 

Ratio  of  LSN  items  researched  in  a  month  compared  to  total  items  received  =  Total 
LSN’s  researched  (sum  LSN  in  Figure  6,  see  Chapter  III)  /  average  number  of  receipts  for 
the  month  (month  sum  of  receipts  from  Figure  7,  see  Chapter  III) 

Therefore,  information  from  Figure  6  and  Figure  7  in  Chapter  III  is  used  to  estimate  the 
average  amount  of  items  and  time  spent  for  the  month  of  July  researching: 

0.0024=  17/7196 

In  other  words,  less  than  1%  of  accepted  items  in  July  were  researched.  Despite  the  low 
percentage,  it  took  about  3.6  hours  to  investigate  the  information  in  order  to  properly 
dispose  of  the  17  items. 


221  min  =17  items  X  13  minutes 

Conversely,  the  verifier  downgrades  to  scrap  the  items  when  too  much  time  is 


spent.  This  generates  a  missed  opportunity  because  the  items  may  actually  have  some 
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recoverable  value.  Moreover,  the  opportunity  cost  is  in  the  form  that  DLA  DS  either 
decides  to  pay  a  contractor  to  scrap  the  items  that  may  still  have  some  recoverable 
monetary  value  if  sold  in  the  commercial  sector,  or  incurs  holding  costs  when  items  are 
stored  in  warehouses.  The  benefit  of  identifying  all  items  upon  arrival  at  DS  provides  a 
leverage  point  to  reuse  or  resale  more  items  rather  than  destroying  them.  Thus,  reselling 
items  returns  revenues  to  the  DOD. 

b.  Problems  that  Create  Backlog 

In  the  previous  section,  two  factors  that  affect  efficiency  were  noted:  improperly 
completed  DD  Forms  1348’s,  and  finding  information  for  obsolete  or  untagged  items  that 
promotes  excess  time  spent  researching  item’s  information.  One  other  issue  the 
researcher  observed  during  the  visit  to  DS  at  Camp  Pendleton  is  that  the  facility  receives 
more  items  than  they  can  process  in  one  day.  The  backlog  accumulates  due  to  the  number 
of  issues  such  as  cross-dock  trucks  that  arrive  each  day,  customers  arrive  with 
unscheduled  appointments  exceeding  demand  for  the  day  and  the  demand  of  items  for 
disposal  exceed  the  facility’s  capacity  to  process  items. 

A  cross-dock  truck  acceptance  is  property  from  another  DLA  DS  site.  The  other 
site  does  not  have  the  capability  to  process  items  due  to  its  limited  authority  to  coordinate 
with  contractors  or  because  it  processes  only  certain  types  of  items,  thereby  transferring, 
or  cross  docking,  it  to  DS  at  Camp  Pendleton.  Items  outside  its  scope  are  delivered  to 
bigger  facilities  that  have  greater  authority  to  dispose  of  items.  Nevertheless,  on  a  daily 
basis,  the  Camp  Pendleton  site  accepts  ten  trucks  from  local  customers  and  an  average  of 
three  cross-dock  trucks  from  other  sites.  Each  truck  is  assigned  an  RCN  that  tracks  the 
number  of  days  it  takes  to  process  the  pallets  and  the  estimated  number  of  Lis  in  them. 
This  observation  was  confirmed  with  the  data  provided  by  the  area  manager. 
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Figure  16.  DLA  DS  at  Camp  Pendleton  Average  RCN  (July  to  October) 


Additionally,  RCN  become  a  nuisance  such  that  the  FTE  may  underestimate  how 
many  items  are  in  the  RCN.  According  to  the  SOP,  the  FTE  estimates  the  amount  of  Lis 
prior  to  stowing  the  items  in  the  backlog  area: 

A  Receipt  Control  Number  (RCN)  is  an  eight  character  alpha/numeric 
sequence  which  indicates  the  Calendar  year,  month,  day,  and  the  number 
of  turn-ins  that  is  manually  generated  via  Distribution  Standard  System 
(DSS).  The  RCN  is  created  when  a  conveyance  is  received  on  site  prior  to 
the  receipt  process  of  Disposal  Turn  In  Documents  (DTID).  The  RCN 
provides  workload  visibility  by  identifying  estimated  line  items  received 
from  individual  turn-in  customers,  a  pallet  count,  and  a  date  and  time  the 
property  was  accepted.  The  sum  of  all  active  RCN  line  counts  will 
determine  and  report  Disposition  Sites  estimated  unprocessed  receipts. 

(DLA  DS,  2013a,  p.  1) 

The  RCN  is  the  first  control  number  tagged  to  the  pallets  in  each  truck  conveyance 
however  a  pallet  does  not  have  a  limit  on  how  many  Lis  it  is  required  to  contain.  One 
pallet  could  have  one  big  bulky  item,  or  it  has  an  average  four  line  items.  The  data  shows 
that  an  RCN  has  an  average  of  13  pallets  resulting  in  an  average  of  50  items  per  RCN.  On 
average  the  DS  at  Camp  Pendleton  can  process  about  300  items  per  day  with  the  current 
staff  capacity,  and  on  average  the  facility  received  8.5  RCN  per  day  resulting  about  425 
items  to  be  processed  per  day.  This  study  covered  almost  a  four-month  period,  and  during 
that  time  there  were  on  average  2,259  items  that  accounted  for  about  47  RCN  or  7.5  day’s 
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worth  of  backlog  that  was  in  the  backlog  stowage  area,  suggesting  that  each  RCN  had 
almost  50  LI,  as  depicted  in  Figure  16.  Therefore,  the  remaining  items  that  are  not 
processed  daily  go  to  the  backlog  for  future  processing.  The  coefficients  are  explained 
with  the  formula: 

125  average  daily  of  items  for  the  backlog  stowage  area  =  425  -  300 

At  the  beginning  of  the  period  of  study  the  oldest  RCN  was  30  days  old  and  at  the 
end  of  the  study  period,  was  45  days.  This  suggests  that  the  ongoing  demand  surpasses 
capacity  to  process  items  for  disposal.  The  logic  of  ongoing  demand  exceeds  the  process 
is  because  the  items  continuously  arrive  so  items  that  arrive  are  processed  plus  the  items 
in  the  backlog,  the  ongoing  issue  mitigates  to  reduce  the  backlog.  However,  if  the  crew 
proposes  only  items  from  the  backlog  without  receiving  any  RCN  it  would  take  about 
7.5  days.  Calculated  by  2,256  average  items  in  backlog  /  309  average  items  DS  at  Camp 
Pendleton  can  process  as  day. 

At  the  beginning  of  July,  the  facility  started  with  1,082  line  items  in  backlog.  By 
the  end  of  July,  the  facility  ended  with  3,072  Lis  in  backlog  and  accepted  a  daily  average 
of  313  items.  How  did  it  triple?  The  disparity  is  noticeable  in  the  first  7  days  of  July.  On 
July  1-3,  the  facility  accepted  about  14  RCN’s  and  the  backlog  doubled,  from  1,082  to 
2,320,  suggesting  that  each  RCN  had  an  average  of  51  Lis.  Additionally,  the  staff  could 
only  completely  process  about  600  items  in  the  2-day  period,  so  the  remainder  that  was 
not  processed  was  kept  in  the  backlog.  This  scenario  is  ongoing  through  the  days  and 
months  so  the  backlog  does  not  reduce  to  zero,  as  illustrated  in  Figure  17.  Furthennore, 
for  the  month  of  July  an  upward  trend  illustrates  demand  exceeded  than  the  amount  of 
items  that  were  processed. 

However,  accepting  Lis  is  not  the  only  task  in  the  focus  of  effort.  The  warehouse 
duties  include  storing,  inventory,  shipping,  customer  pick-up,  rearrangement  of  backlog, 
cleaning,  and  scrap  management.  In  addition,  the  site  provides  property  disposal  classes 
to  all  local  units  and  accepts  the  property  from  four  other  cross-dock  sites  for  processing. 
Considering  all  these  factors  suggests  a  growing  problem  that  has  potential  leading 

bottleneck,  backlogs,  and  increased  disposition  time. 
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Figure  17.  DLA  DS  at  Camp  Pendleton  Day’s  Worth  of  Backlog  Items  for  July  2014 

Figure  17  depicts  all  DS  working  days  for  days’  worth  of  backlog  items  for  July. 
The  calculation  is  the  result  of  using  the  backlog  ratio  formula  such  that  the  calculation 
for  1  July  and  so  on  is  the  result  from  this  computation: 

2.3  days’  worth  of  backlog  items  =  (1082  backlog  items  /  486  items  processed  on  1  July) 

The  backlog  ratio  steadily  glows  throughout  the  month  of  July,  reaching  an 
average  of  6.9  days’  worth  of  items  in  the  backlog  for  the  month,  depicted  in  red  above 
number  32.  Equally,  day  23  is  a  day  of  concern  because  it  reached  over  22.5  days’  worth 
of  working  items  from  the  backlog  stowage  area,  suggesting  that  a  double  amount  of 
items  were  received  compared  to  the  previous  day.  Nonetheless,  for  the  four-month 
period,  the  trend  is  downward,  as  illustrated  in  Figure  18,  resulting  in  an  average  of  7.5 
day’s  worth  of  backlog  items.  Arguably,  this  numbers  suggest  a  backlog  of  days  worth  of 
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items  for  proper  disposition  is  ongoing  and  management  needs  to  find  a  solution  to 
reduce  it  and  eliminate  backlogs. 


Days'  Worth  of  Backlog 


Day 

Figure  18.  DLA  DS  at  Camp  Pendleton  Day’s  Worth  of  Backlog  Items 

from  July  to  October  2014 


Inverse  trend  relationship  between  processed  versus  backlog  items.  The  data 
shows  a  trend  that  showcases  underlying  indicators  of  an  inverse  effect  that  an  increase  of 
backlogged  items  results  in  the  decrease  of  processed  items  and  vice  versa.  The  trend  is 
observed  when  at  the  beginning  of  the  week  more  items  are  accepted  affecting  items 
processed  to  fewer  numbers.  Conversely,  fewer  items  are  accepted  thus  creating  less 
backlog  and  more  items  are  processed  thus  increasing  the  amount  of  processed  items. 
This  inverse  relationship  is  clearly  depicted  in  Figure  19,  suggesting  that  in  order  to 
reduce  the  backlog  fewer  items  must  be  accepted  or  productivity  must  be  increased  to 
equal  out  both  components.  Additionally,  the  backlog  ratio  compared  to  the  number  of 
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items  processed  is  almost  triple,  indicating  that  DS  at  Camp  Pendleton  cannot  keep  up 
with  more  demand  than  what  they  can  process.  DS’s  property  acceptance  process  is  not 
enabled  with  technology  and  automation,  so  it  does  not  deliver  efficient  or  effective 
outcomes  when  benchmarked  with  comparable  commercial  practices. 


Inverse  Relationship  between  Backlog  and  Processed 

Items 
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Figure  19.  DLA  DS  at  Camp  Pendleton  Inverse  Relationship  Pattern  between 
Backlog  and  Processed  Items  from  July  to  October  2014 
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Figure  20.  Two  Military  Tents  with  Backlog  Items 
Note.  The  pictures  were  taken  by  the  researcher  in  DLA  DS  at  Camp 
Pendleton,  California,  in  July  2014 


Figure  20  shows  two  pictures  of  two  dome  tents  outside  the  warehouse  at  DLA 
DS,  Camp  Pendleton.  When  the  picture  was  taken  there  were  about  47  RCNs  with  an 
estimated  2,313  items  that  accounted  for  about  7.5-backlog  ratio  in  days’  worth  of  items 
that  are  awaiting  disposal. 

c.  Analyzing  Backlog  using  Analytical  Model  to  Resolve  Backlog 

A  backlog  of  items  that  are  waiting  for  future  processing  is  a  phenomenon  created 
by  unprocessed  items  that  have  been  received  but  are  not  processed  or  are  in  the  process 
of  being  completed  because  the  number  of  items  arrival  exceed  the  employees’  capacity 
to  process.  Unpredictability  of  items  arrival  and  variability  of  service  processing  time 
leads  to  backlog  or  queues.  Understanding  and  learning  the  queuing  is  an  important  area 
in  operations  management  (Jacobs  &  Chase,  2010).  For  DLA  DS  at  Camp  Pendleton, 
item-processing  demand  vary  by  the  amount  of  items  processed  per  employee  and  often 
the  amount  of  items  exceeds  normal  capacity  that  the  six-crew  employee  at  Camp 
Pendleton  can  handle.  However,  the  arrival  of  items  can  be  controlled  in  a  variety  of 
ways.  For  example,  DLA  DS  can  reduce  the  amount  of  appointments  per  day,  thereby 
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reducing  the  amount  of  items  accepted,  they  can  limit  the  number  of  items  per  pallet  so 
that  they  only  receive  about  300  items  per  day,  or  extend  the  hours  employees  work  but 
only  process  items  in  the  queue  and  not  accept  more  items  during  the  extended  hours.  For 
the  employees,  DLA  DS  can  affect  service  time  by  adding  more  employees,  or 
implement  machines  to  automate  the  process  flow  to  increase  productivity. 

According  to  Jacobs  and  Chase  (2010),  the  queuing  system  consists  of  three 
components:  (1)  the  source  of  items  and  the  way  items  arrive,  (2)  the  servicing  system 
such  as  servers  or  employees,  and  (3)  the  condition  the  items  exit  the  system  (p.  40).  By 
calculating  the  three  queuing  components  one  can  formulate  the  number  of  employees 
DLA  DS  at  Camp  Pendleton  are  needed  to  reduce  and  prevent  days’  worth  of  backlog. 
Therefore,  to  formulate  the  amount  of  employees  are  needed  at  DLA  DS  the  coefficients 
in  the  following  formula  were  utilized  in  a  spreadsheet,  Figure  21,  designed  to  compute 
the  characteristics  of  employees  in  the  multiple-server  queuing  model  using  arrival  rate  of 
items  and  service  rates  each  employee  processes. 

The  queuing  model  illustrated  in  Figure  21  used  results  from  previous  section 
with  regard  to  averages  of  items  received  at  DLA  DS  at  Camp  Pendleton,  California, 
average  processed  items  per  employee  per  hour,  and  number  of  employees  in  the  items 
disposal  crew  at  DLA  DS  Camp  Pendleton,  California.  That  means  that  on  a  daily  basis 
the  Camp  Pendleton  facility  received  an  average  of  425  items  for  disposal;  therefore  per 
hour  they  accepted  about  65  items  per  hour  and  an  employee  can  process  7.5  items, 
suggested  by  the  DLA  DS  study  (2013)  and  the  Camp  Pendleton  facility  (2014)  had  a 
six-employee  crew  for  item  disposal  during  the  period  of  the  study. 
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Analysis:  Number  of  Employee  Required  at  DLA  DS, 

Camp  Pendleton,  CA 

Assumtions:  Multiple  servers,  Infinite  population,  Poisson  arrival,  Exponential  service  time, 

Unlimited  waiting  warehouse  size 

Yellow  cells  need  user  inputed  values 

Inputs 

Coefficient  unit: 

Unit  of  time 

Hour 

Arrival  rate  (lambda) 

65.38461538 

Items  per  hour 

Service  rate  (mu) 

7.5 

Items  per  hour 

Number  of  identical  employees  (server  [s]) 

9 

Employees 

Summary  measures 

Notes: 

Average  utilization  rate  of  server 

96.9% 

Capacity 

Average  number  of  items  waiting  in  backlog  (Lq) 

27.64978 

Items 

Average  number  of  items  in  system  (L) 

36.36773 

Items 

Average  time  waiting  in  line  (Wq) 

0.42288 

Hour 

Average  time  in  system  (W) 

0.55621 

Hour 

Probability  of  no  customers  in  system  (PO) 

0.00003 

Probability  of 
empty  system 

Probability  that  all  servers  are  busy 

89.5% 

Percentage  of 
items  that  wait  in 
backlog 

Probability  that  at  least  one  server  is  idle 

10.5% 

Percentage  of 
items  that  don't 
wait  in  backlog 

Figure  2 1 .  Spreadsheet  of  Analysis  the  Number  of  Employees  Required  at  DLA  DS 

in  Camp  Pendleton,  California 


Figure  21  is  an  Excel  template  for  using  analysis  with  the  queuing  model  to 
choose  the  number  of  employees  for  DLA  DS  at  Camp  Pendleton,  California.  The 
spreadsheet  was  designed  to  compute  characteristics  of  multiple  employees,  using  the 
average  arrival  rate  of  items  per  hour,  and  service  rate  each  employee  can  process  per 
hour.  The  output  coefficients  are  the  mean  time  of  items  arrival  at  per  hour  during  an 
expected  6.5-hour  working  per  day,  and  the  mean  time  items  processed  for  a  crew  of 
employees  per  hour  during  an  expected  6.5-hour  per  day. 

Table  4  illustrates  the  results  from  the  queuing  model  with  changing  employees 

from  six  up  to  ten.  With  eight  employees  one  can  see  significant  improvement  to  the 
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probability  to  decrease  backlog  from  255%  to  135%.  Adding  the  ninth  employee 
provides  a  97%  capacity  meaning  that  the  facility  can  keep  up  with  demand  by 
processing  all  received  items  and  have  a  3%  idle  of  down  time,  which  will  be  initially 
used  to  reduce  the  7.5  days’  worth  of  backlog.  Therefore,  with  nine  employees  the 
facility  provides  a  97%  capacity  in  term  of  processing  items  upon  arrival  to  mitigate 
backlog  in  the  future.  On  average  DLA  DS  received  425  items  a  day  and  with  a  crew  of 
six  the  DLA  DS  at  Camp  Pendleton  processed  about  300  items;  thus,  on  a  daily  basis 
about  125  items  are  placed  in  the  backlog  to  process  them  on  the  first-in  first-out  basis. 
The  facility  had  7.5  days’  worth  of  backlogs;  therefore  with  nine  employees  the  facility 
will  slowly  reduce  the  backlog  and  be  at  capacity  with  relations  to  items  arrival,  because 
the  facility  will  be  able  to  process  450  items  per  day  and  about  425  items  are  accepted. 


Table  4.  Sensitivity  Analysis  of  Number  of  Employees  to  Reduce  Backlog 


Sensitivity  Analysis:  Number  of  Employee 
Required  at  DLA  DS,  Camp  Pendleton,  CA 

Employee  (s) 

Facility 

Capacity 

Percent  Items 
in  Backlog 

Percent  Items 
not  in  Backlog 

6 

145% 

255% 

- 

7 

125% 

185% 

- 

8 

109% 

131% 

- 

9 

97% 

90% 

10% 

10 

87% 

59% 

41% 

d.  Summary 

The  process  flow  is  veiy  lengthy,  and  DLA  DS  needs  to  manage  it  in  an  average 
of  42  days  in  order  to  provide  an  opportunity  to  RTD  items  to  branches  of  service,  federal 
agency,  or  local  agency.  DLA  DS  needs  reduce  backlogs,  increase  efficiency  and  take 
advantage  of  opportunities.  As  seen  hi  this  analysis,  there  are  steps  in  the  process  flow 
that  have  potential  to  create  a  bottleneck.  This  section  examined  the  three  steps  in  the 
items  process  that  are  areas  of  concern.  The  results  show  that  there  are  four  areas  of 
concern  that  contribute  to  creating  days’  worth  of  backlog  items.  First,  improperly  filled 
DD  1348  provides  for  additional  time  to  spend  conducting  research  in  order  to  fill  the  DD 
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1348  for  further  processing.  Second,  verifiers  spend  time  to  conduct  research  on  items 
that  are  not  labeled  or  documented  on  a  DD  1348.  About  less  than  1%  of  accepted  items 
at  are  unlabeled.  However,  the  1%  of  unlabeled  items  is  not  the  main  problem  of  days’ 
worth  of  backlog  items,  the  third  problem  can  be  safely  argued  that  DLA  DS  at  Camp 
Pendleton  accepts  more  items  than  what  it  processes,  resulting  in  an  average  over  2000 
items  in  the  backlog  stowage  area.  DLA  DS  at  Camp  Pendleton  is  limited  to  processing 
about  300  items  per  day  but  it  received  an  average  425  per  day  in  an  average  of  8.5  RCNs 
per  day,  leaving  the  remainder  to  add  to  the  backlog;  therefore,  the  demand  exceeded 
the  amount  of  items  that  were  processed.  Similarly,  fourth,  RCNs  are  not  limited  to  the 
amount  of  items  that  will  fit  in  a  pallet  or  tri-box,  such  that  an  RCN  can  have  about 
50  items.  The  results  from  the  queuing  theory  suggests  that,  with  nine  employees,  the 
DLA  DS  at  Camp  Pendleton  will  keep  up  with  demand  by  processing  all  received  items 
and  have  a  3%  idle  time,  which  will  be  utilized  to  reduce  the  7.5  days’  worth  of  backlog. 

DLA  DS  made  progress  in  reducing  the  lead-time  on  the  process  flow  by 
implementing  the  Lean  Six  Sigma  in  2008;  however,  additional  changes  need  to  be 
created  in  order  to  improve  the  lead-time.  In  general,  these  challenges  are  corrected  with 
adequate  automated  processes  and  the  focus  of  this  study  is  addressing  inefficiency  to 
correct  it  with  the  implementation  of  technology.  With  that  in  mind,  the  next  section 
assesses  object/shape  recognition  technology  COTS  sectors. 

B.  ASSESSMENT  OF  OBJECT/SHAPE  RECOGNITION  TECHNOLOGY 

MATURITY 

1.  Overview 

The  Technology  Readiness  Assessment  (TRA)  is  designed  to  allow  program 
managers,  acquisitions  personnel,  and  decision-makers  to  understand  the  risk,  cost,  and 
time  associated  with  maturing  technologies.  This  section  of  this  study  reports  the  results 
of  the  assessment  results  of  three  different  companies’  object/shape  recognition 
technology.  The  three  companies  of  interest  are  listed  in  Figure  22  with  each  company’s 
description.  The  assessment  follows  the  Department  of  Defense  Technology  Readiness 
Assessment  Guidance  (ASD[R&E],  2011),  and  for  the  purpose  of  this  study  the 
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researcher  adopted  and  customized  the  Department  of  Energy’s  (2012)  template  for 
assessment  to  make  it  suitable  for  assessing  object/shape  recognition  technology  systems. 
The  assessment  consists  of  separate  reports  for  each  companies’  technology  that  include 
(1)  an  overview  of  the  company’s  object/shape  recognition  technology  profde;  (2) 
technology’s  goal;  (3)  benefits;  (4)  critical  area(s)  of  challenge;  (5)  qualitative  summary 
of  the  TRL;  and  (6)  a  statement  of  relevancy  to  DLA  DS.  Finally,  the  section  concludes 
with  a  summary  of  the  strengths  and  weaknesses  of  object/shape  recognition  technology. 
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Object  Shape  Recognition  Companies  and  their 

Programs 


Cognex  delivers  the  widest  range  of  image-based  factory  automation  and 
distribution  products  to  help  companies  like  you  ensure  error-free 
production,  lower  costs,  and  manage  your  supply  chain: 

Barcode  readers  to  ensure  part  and  product  traceability 

Vision  systems  and  software  for  quality  inspection  and  automation 

Vision  sensors  to  detect  part  features  and  verify  assembly 

Laser  profilers  and  software  for  3D  inspection  to  further  optimize  quality 

Source:  www.cognex.com 


When  business  leaders  and  innovators  like  you  engage  with  Imaginestics  and 
discover  the  VizSeekS  shape-search  engine,  their  first  question  is  often: 

"Can  it  do  this?"  Instinctively,  the  answer  is  always,  "Yes." 


The  VizSpace  supply-chain  sourcing  network  and  the  evolution  of  clientele 
licensing  our  shape-search  technology  are  all  prime  examples  of  true 
American  ingenuity.  If  you  can  envision  a  function  for  shape  search,  inside 
the  VizSpace  network  or  out.  the  Imaginestics  development  team  is  available 
to  partner  with  you  and  engineer  a  new  application  that  meets  your  group's 
unique  requirements. 


We  have  a  creative,  technically  innovative  team  adept  at  taking  your 
imagined  solution  and  making  it  a  reality.  The  possibilities  for  shape  search 
are  limitless  -from  determining  costto  configuring  products.  Get  in  touch 
with  us  and  give  us  your  idea.  The  possibilities  are  limitless! 

Source:  wwwjmaginestics.com 


Augmented  Reality 

RMD  is  developing  a  revolutionary  training  and  enabling  technology  based  on 
augmented  reality.  It  isextremely  valuable  for  speeding  training  and  repairs, 
detecting  and  preventing  errors,  saving  money  and  time,  and  improving 
safety. 

Source:  www.rmdinc.com 


Figure  22.  Three  Companies  with  Recognition  Technology  Programs 
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2.  Cognex 

Overview:  Cognex ’s  technology  is  a  belt-driven  single  file  carousel  that  identifies 
items  by  scanning  bar  code  tags.  The  items  are  associated  with  a  database  mainly  by 
using  the  serial  numbers  on  the  item’s  tag.  Cognex  manufactures  the  hardware  like  the 
image  capture  devices  and  they  partner  up  with  companies  that  offer  integrated  solutions 
to  connect  Cognex  hardware.  However,  for  large  multi-million-dollar  project  Cognex 
offers  direct  integration  to  innovate  a  system.  Consequently,  Cognex  does  not  have  a 
prototype  application  to  qualify  for  DLA  DS  requirement,  but  its  current  concept  of 
processing  items  is  applicable  to  DLA  DS. 

Cognex  Technology’s  Goal:  Currently,  Cognex’s  technology  is  used  in 
companies  where  specific  item  precision  is  required.  For  example  Cognex  technology 
detects  defects  with  sensors,  3-dimensional  (3D)  lasers,  scanners  and  cameras  in  order  to 
improve  production  productivity  at  a  higher  speed. 

Benefits:  The  technology’s  approach  is  to  provide  efficiency  and  effectively 
manufacture  or  inventory  items.  The  industries  that  benefit  from  Cognex  technology  are 
automotive,  food  and  beverage,  and  electronics  as  depicted  in  Figure  23.  With  regard  to 
DLA  DS,  Cognex’s  concept  that  can  be  applicable  currently  is  not  in  a  COTS  form  that 
DLA  DS  can  plug  and  start  to  operate. 

Critical  Area  of  Challenge:  Critical  technology  challenges  related  to  meeting 
DLA  DS  requirements  include  the  fact  that  the  Cognex  machines  are  specific  to  a 
company’s  design  and  needs,  so  a  prototype  to  fit  DLA  DS  needs  will  need  to  be 
reengineered  from  the  ground  up.  What  this  means  is  that  operational  equipment  intended 
to  identify  an  object  without  bar  codes  technique  is  not  available  at  Cognex.  However, 
Cognex  has  the  technology  and  partners  to  conduct  R&D  in  order  to  custom  make  a 
technology  for  DLA  DS. 

Qualitative  Summary  of  Cognex’s  Technology  (TRL):  A  technology  that  is 
ready  with  DS’  specification  and  to  be  operated  at  DS  facility  or  similar  environment  has 
not  been  created.  Clearly,  Cognex  will  be  required  to  engineer  and  integrate  elements 
from  partners  that  would  require  technology  maturation;  therefore,  Cognex’s  technology 
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values  in  the  TRL  scale  2  (see  Table  5  for  the  TRL  definition).  Currently,  Cognex  has 
made  machines  that  operate  at  specific  owners  requirement,  so  they  have  the  concept  and 
principle  formulated  relevant  to  DLA  DS  requirement.  However,  technology  for  DLA  DS 
would  be  considered  to  be  in  a  limited  paper  study  such  that  its  current  system  would  be 
left  for  speculation  to  be  applicable  as  it  is;  therefore,  further  maturation  would  be 
required  under  an  investment  program  and  contract. 


Table  5.  Description  of  Technology  Readiness  Level  2  (from  USAF,  2010) 


TRL 

Definition 

Description 

2 

Technology  Concept  and/or 
Application  Formulated 

Invention  begins.  Once  basic  principles  are 
observed,  practical  applications  can  be  invented. 
The  application  is  speculative  and  there  is  no  proof 
or  detailed  analysis  to  support  the  assumption. 
Examples  remain  limited  to  paper  studies. 

Statement  of  Relevancy  to  DLA  DS:  Cognex  explicitly  described  that  a  project  of 
large  scale,  such  as  full  automation  with  carousel  belts,  scanners,  and  image  capture 
devices,  would  be  worth  pursuing.  Meanwhile,  a  small  project,  such  as  a  mobile  device 
with  image  capture  apparatus,  would  be  an  undertaking  for  one  of  Cognex’s  partners  and 
eventually  Cognex  would  mature  it  to  a  grand  scale. 

Figure  23  illustrates  Cognex  technology  that  is  designed  to  scan  and  read  bar 
codes  while  the  items  move  on  a  carousel  system.  Similarly,  the  laser  and  scanner  can 
detect  anomalies  and  read  codes  and  discrepancies  with  items  specifications. 
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DataMan  Logistics  Barcode  Reader  Applications 


Tote  Scanning  Single-sided  Scanning  Carton  Coding  Print  &  Apply 


Optical  Character  Recognition  Applications  for  3D  Laser  Profiling  J  Presence/Absence  inspection 


'Z  Read  DOT  codes  on  automobile  tires 

Food  and  Beverage 

V 

▼  Contrast  independent  inspection 

V  Generate  3D  images  of  embossed 

Electronics 

or  raised  characters 

V fcrfy  d(Kt  hecM  n  n 


Figure  23.  Cognex  Product  Identification  Machines 
Note.  Picture  from  Cognex  Pamphlet  and  Cognex  YouTube  video 


Figure  24  illustrates  a  large-scale  supply  chain  logistics  warehouse  operated  with 
Cognex  technology.  The  technologies  included  are  scanners,  and  laser  and  image  capture 
devices  to  efficiently  and  effectively  control  inbound  and  outbound  inventory. 
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©Inbound 


O  Order  Fulfillment 


Cognex  readers  have: 

•  The  highest  read  rates  •  Performance  feedback  •  No  moving  parts 


Figure  24.  Cognex  Large  Scale  Machines  (from  Product  Cognex,  n.d.) 

3.  Imaginestics 

Overview:  Imaginestics  has  developed  a  technology  that  is  an  application  for 
mobile  devices,  which  includes  phones  and  tablets  for  both  Apple  and  Android  operating 
systems.  The  technology  is  designed  to  find  shapes  and  objects  that  are  similar  to  a  given 
visual  input,  labeled  by  Imaginestics  Smart  Shape  Technology  and  trademarked  as 
VizSeek®.  A  picture  captured  by  the  mobile  device  is  used  to  search  shapes  in  a 
VizSeek®  database.  The  image’s  shape  is  associated  with  an  indexed  database  to  provide 
feedback.  The  mobile  device  conducts  the  search  on  VizSeek®  which  requires  either 
Internet  or  an  intranet  (behind  the  firewall)  to  access  the  database.  To  create  the  indexed 
database,  VizSeek®  extracts  and  archived  shapes  and  text  from  images,  documents,  and 
other  sources  of  database.  Imaginestics  main  contribution  to  the  shape  recognition 
technology  is  to  extract  the  information  by  normalizing  the  data  so  that  information  is 
indexed  from  the  content,  such  as  diagrams  and  images  from  manuals,  publications,  and 
so  forth.  Imaginestics  technology  is  capable  of  searching  images  from  two  and  three- 
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dimensional  models  (see  Figures  15  and  16).  This  means  that  it  recognizes  the  image 
from  any  angle  and  fonnat  rather  than  recognizing  an  image  to  an  image.  Imaginestics 
was  a  winner  of  the  Indiana  Entrepreneurship  Award.  The  merit  of  the  award  is 
outstanding,  but  what  is  of  importance  to  this  assessment  is  the  fact  that  the  DOD  funded 
the  research,  so  Imaginestics  understands  the  DOD  acquisitions  process.  The  award 
citation  states  the  following: 

Imaginestics’  vision  is  to  organize  and  market  industrial  manufacturer’s 
product  and  service  infonnation  and  make  it  visually  searchable  and  useful 
worldwide.  As  the  first  step  in  making  industrial  manufacturer’s  product 
and  service  information  visually  searchable... developing  2D  shape 
matching  technology  and  unique  intranet  search  engine...  [and]... 
developing  the  3D  shape  search  algorithm  at  Purdue  University.  [Then] . . . 
developed  VizSeek,  the  world’s  first  online  visual  search  engine  for 
manufacturing.  In  addition,  they  also  worked  on  value  added  solutions 
such  as  VizAdvise,  VizCompare  and  VizConfig  to  make  the  search  results 
obtained  useful  for  the  search  user.  Imaginestics  received  funding  support 
from  National  Science  Foundation,  Department  of  Defense  and  Indiana 
Economic  Development  Corp.  to  conduct  research  and  development  of 
these  solutions.  (“Indiana  Entrepreneurial  Leadership  Awards  Winners,” 
n.d.) 

Imaginestics  Technology’s  Goal:  Imaginestics’  technology  is  used  to  organized 
information  such  as  shapes  of  objects  so  that  it  is  searchable  using  a  mobile  device  and 
feedback  with  the  images’  infonnation  is  provided  in  seconds  under  the  condition  that  the 
database  has  the  infonnation  indexed. 

Benefits:  The  technology’s  approach  is  to  provide  infonnation  from  an  image. 
The  industries  that  benefit  from  Imaginestics’  technology  are  aviation  and  supply  chain 
management.  Currently,  Lockheed  Martin  uses  the  technology  to  conduct  research  and 
make  costing  estimates.  With  regard  to  DLA  DS,  Imaginestics’  technology  is  applicable. 
As  a  COTS  technology  DLA  DS  will  require  further  maturation  to  be  applicable  to  DLA 
DS  requirement.  For  example  Imaginestics  is  capable  of  indexing  information  from 
converted  images  in  optical  character  recognition  (OCR)  format  such  as  printed  manual 
in  order  to  digitized  it  and  index  it  in  the  VizSeek®  database.  This  modification  will 
require  the  buildup  of  a  database  specifically  for  DLA  DS.  Another  benefit  from 
Imaginestics  is  that  the  mobile  device  that  captures  the  image  can  be  used  to 
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consequently  upload  the  same  picture  in  DLA  DS  liquidation  website  to  resale  the  items 
rather  than  contracting  a  photographer. 

Critical  Area  of  Challenge:  Critical  technology  challenges  related  to  meeting 
DLA  DS  requirements  include  the  fact  that  a  database  will  have  to  be  built.  Additionally, 
because  the  system  operates  via  the  Internet  and  Apple  Inc.  devices  the  challenge  relates 
to  getting  approval  from  the  Defense  Information  System  Agency  (DISA)  with  respect  to 
information  assurance  (IA)  credentials.  However,  the  credibility  package  is  part  of  the 
acquisitions  process. 

Qualitative  Summary  of  Imaginestics’  Technology  (TRL):  Imaginestics 
technology  is  the  closest  technology  that  is  ready  with  DS  specification  and  to  be 
operated  at  a  DS  facility,  however  is  has  not  been  operated  in  a  DS  similar  environment 
under  DISA  IA  requirements.  Imaginestics’  shape  recognition  technology  is  being 
utilized  by  Lockheed  Martin,  so  technology  components  relevant  to  DLA  DS  requirement 
is  known  to  work.  However,  the  database  aspect  of  the  technology  will  be  required  to  be 
engineered  and  to  integrate  elements  from  the  current  prototype,  which  would  require 
maturation;  therefore,  Imaginestics’  technology  values  in  the  TRL  scale  between  4  and  5 
(see  Table  6  for  the  TRL  definitions).  Currently,  Imaginestics  has  made  machines  that 
operate  at  specific  owner’s  requirement,  so  it  has  the  concept,  principle,  and  prototype 
formulated  relevant  to  DLA  DS  requirement.  The  system  hardware  and  software 
integration  has  progressed  beyond  the  laboratory  phase. 
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Table  6.  Descxiption  of  Technology  Readiness  Levels  4  and  5 

(from  USAF,  2010) 


TRL 

Definition 

Description 

4 

Component 

and/or 

Breadboard 
Validated  in 
Laboratory 
Environment 

Basic  technological  components  are  integrated  to  establish 
the  feasibility  that  the  pieces  will  work  together.  This  is 
relatively  low  fidelity  compared  to  the  eventual  system. 
Examples  include  integration  of  ad  hoc  hardware  in  a 
laboratory. 

5 

Component 

and/or 

Breadboard 
Validated  in 
Relevant 
Environment 

Fidelity  of  breadboard  technology  increases  significantly. 

The  basic  technological  components  are  integrated  with 
reasonably  realistic  supporting  elements  so  that  the 
technology  can  be  tested  in  a  simulated  environment. 
Examples  include  high-fidelity  laboratory  integr  ation  of 
components. 

Statement  of  Relevancy  to  DLA  DS:  Imaginestics  has  been  approved  for  IA 
approval  for  another  technology  that  is  contracted  under  the  Navy  and  Office  of  the 
Secretary  of  Defense  (OSD);  therefore  Imaginestics  understands  the  DOD  acquisitions 
processes  and  IA  approval  request  process. 

Figures  25  and  26  illustrate  the  search  and  results  of  a  compressor,  respectively. 
The  image  of  the  compressor  in  Figure  25  was  matched  with  an  image  of  a  compressor 
from  government  liquidation  website.  The  results  illustrated  in  Figure  26  is  the  interface 
from  VizSeek®  depicting  the  feedback  information  of  the  compressor,  such  as  the  cost, 
NSN,  DMIL  code  and  DTID  number  related  to  the  order  tracking  number  entered  by 
DLA  DS  when  the  compressor  was  processed  for  resale. 
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Search  Results -Compressor,  Reciprocating 

*£>  VteSee*  X 


A  photo  of  a  compressor  is  used  to  search  and  find  matching  DLA  compressors. 


Figure  25.  VizSeek®  Application  Matches  Compressor  Image  with  Database 
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Figure  26.  VizSeek®  Results  with  Image  and  Pertinent  Infonnation 
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4. 


RMD 


Overview:  RMD  created  a  technology,  called  the  automated  digital  recognition 
technology  (ADIRT),  which  is  focused  on  the  artificial  intelligence  perspective.  The 
technology  is  designed  to  capture  images  but  the  user  verifies  the  information  the  first 
time  the  image  is  introduced  to  the  ADIRT.  The  subsequent  time  when  the  image  is 
presented  ADIRT  automatically  recognizes  it  to  processes  it  for  identification  and 
provides  feedback.  RMD  built  a  prototype  for  the  Marine  Corps.  The  Marine  Corps 
initially  funded  the  R&D;  however,  under  the  budgetary  cuts  the  program  was  no  longer 
funded.  Consequently,  RMD  has  not  matured  the  technology  any  further;  however, 
RMD’s  technology  qualifies  within  the  DLA  DS  requirement.  RMD’s  technology 
concept  of  processing  items  is  applicable  to  DLA  DS,  but  requires  further  funding  for 
maturation.  RMD  estimates  a  two-year  period  to  provide  a  prototype  to  DLA  DS. 

RMD  Technology’s  Goal:  ADIRT  under  its  current  state  was  designed  to  support 
electronic  technicians  to  automatically  recognize  a  circuit  board  retrieving  schematics  for 
repairs.  ADIRT  uses  a  laser  projector  to  point  at  the  part  in  the  circuit  board  so  the 
mechanic  conducts  the  repairs  to  the  component  in  the  circuit  board.  This  enables  the 
technician  to  identity  and  locate  unlabeled  components  and  part  numbers  easily,  correlate 
them  with  documentation,  and  communicate  with  databases  across  platforms  and 
organizations  (RMD,  2014). 

Benefits:  The  technology’s  approach  is  to  capture  knowledge  in  the  form  of 
artificial  intelligence  so  that  any  technician  uses  the  technology  to  guide  them  when 
repairing  the  circuit  board.  The  Marine  Corps  concept  was  to  retain  in  a  database  the 
knowledge  of  well-training  electronic  technician  and  pass  on  the  knowledge  to  any  junior 
Marine.  Given  that  ADIRT  is  designed  to  remember  procedure  in  accordance  with 
technical  manuals,  when  the  technician  fixes  a  circuit  board,  ADIRT  provides  the 
potential  to  capture  any  knowledge  to  turnover  for  continuity. 

Critical  Area  of  Challenge:  Critical  technology  challenges  related  to  meeting 
DLA  DS  requirements  include  the  fact  that  the  initial  prototype  was  built  for  a  different 
purpose.  Additionally  because  the  system  operates  via  the  Internet  and  has  the  capability 


79 


to  operate  in  a  mobile  device,  the  challenge  relates  to  getting  approval  from  the  DISA 
with  respect  to  IA  credentials.  However,  the  credibility  package  is  part  to  acquisitions 
process.  Additionally  the  current  prototype  is  not  in  a  mobile  device  state.  The  current 
prototype  is  a  computer  with  a  camera  and  laser  projector,  so  the  challenge  is  to  reduce 
the  size  to  a  mobile  device,  which  calls  for  further  maturation. 

Qualitative  Summary  of  HMD’s  Technology ’  (TRL):  RAID  will  be  required  to 
engineering  and  integrating  elements  from  its  current  prototype,  ADIRT,  which  would 
require  further  R&D  for  the  technology  to  mature;  therefore,  RDM’s  technology  values 
in  the  TRL  scale  2  (see  Table  7  for  the  TRL  definition).  Currently,  RDM  has  made  a 
prototype  that  operated  for  the  Marine  Corps  requirement  of  training  technicians  to 
conduct  repairs,  and  the  technology  detects  and  prevents  errors.  RMD  has  the  concept 
and  principle  formulated  relevant  to  the  DLA  DS  requirement.  However,  to  make  RMD’s 
technology  frilly  relevant  to  DLA  DS,  the  technology  is  considered  to  be  in  a  limited 
paper  study  such  that  RMD  current  system  is  beyond  speculation  to  be  applicable  as  it  is; 
therefore,  further  maturation  would  be  required  under  an  investment  program  and 
contract. 


Table  7.  Description  of  Technology  Readiness  Levels  2 

(from  USAF,  2010) 


TRL 

Definition 

Description 

2 

Technology  Concept  and/or 
Application  Formulated 

Invention  begins.  Once  basic  principles  are 
observed,  practical  applications  can  be  invented. 
The  application  is  speculative  and  there  is  no 
proof  or  detailed  analysis  to  support  the 
assumption.  Examples  remain  limited  to  paper 
studies. 

Statement  of  Relevancy  to  DLA  DS:  RMD  explicitly  explained  that  the 
technology  was  designed  for  mechanical  maintenance  rather  than  for  supply-chain 
management.  According  to  RMD  it  will 

deliver  a  prototype  to  DLA  in  the  tenth  month  of  year  one  for  testing  at 
their  facilities.  In  year  two  of  the  program,  RMD  will  deliver  a  more 
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advanced  prototype  in  month  22  of  the  24-month  program  for  test  and 
validation  at  the  DLA”  (RMD,  2014). 

Therefore,  RDM’s  technology  is  not  ready  for  the  near  future  and  would  required 
excess  funding  to  mature  it. 

Figure  27  illustrates  RMD’s  technology  that  is  designed  to  scan  and  read 
schematics  while  the  technician  makes  repairs  to  the  circuit  board.  The  laser  projector  is 
designed  to  pinpoint  which  component  on  the  circuit  board  to  repair.  The  technology  is 
designed  to  leam  the  process  the  first  time  it  is  introduced  with  a  circuit  board.  The 
subsequent  time  when  the  same-type  of  circuit  board  it  repaired  the  system  navigates  the 
technician  to  conducts  the  repair  to  the  component  as  previously  conducted. 


Figure  27.  RMD  Product  Identification  Machines  (picture  from  rmdinc.com) 

5.  Strengths  and  Weakness  of  Object/Shape  Recognition  Technology 

The  root  causes  for  automating  a  company  with  technology  is  to  cut  costs,  target 


inefficiency,  and  control  breakdowns  in  managing  inventory,  and  capture  data  for 
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analytics  purpose.  Technology  presents  strength  and  weakness,  including  strengths  with 
standardization,  productivity,  inventory  control,  and  management.  Conversely, 
technology  presents  weaknesses  with  upfront  capital  investment,  reliance  on  the 
information  and  information  assurance,  and  equipment  fragility. 

a.  Strengths 

1 .  Standardization — technology  ensures  uniformity  and  conformation  across 
the  board. 

2.  Productivity —  technology  can  reduce  time  in  identifying  items,  thereby 
increasing  productivity. 

3.  Benefit — the  benefits  will  outweigh  the  initial  investment  cost,  in  the  long 
run,  usually  after  the  payback  period  then  benefit  grows  by  providing  time 
and  labor  saving  earnings. 

4.  Accuracy — error  is  marginal,  when  proper  infonnation  is  indexed  to  return 
correct  feedback.  A  database  program  provides  the  ability  to  conduct  data 
mining  for  analytics.  Analytics  improves  operational  efficiency  by 
facilitating  effectiveness  of  processing  items. 

5.  Operate — ease  of  operation  with  minimal  training. 

b.  Weaknesses 

1 .  Cost — initial  upfront  investment  cost  and  maintenance  cost.  Additionally, 
technology  has  a  double-edged  sword,  such  that  favorable  benefits  are 
gained  upon  the  recovery  of  the  funds  expended.  However,  the  technology 
depreciates  and  wears  to  the  point  of  affecting  its  performance — 
ultimately  becoming  inoperable.  Therefore,  replacing  it  adds  another 
round  of  cost. 

2.  Feedback — reliance  on  information  keyed  in  the  database.  In  other  words, 
the  feedback  accuracy  relies  on  the  manually  entered  information  in  the 
database,  unless  it  is  an  artificial  intelligence  that  is  adaptive  and  self- 
learning.  Nonetheless,  the  feedback  could  be  unreliable  due  to  incorrect 
data  in  the  database.  The  trend  is  counteracted  if  the  wrong  information  is 
entered  where  the  focus  should  be  elsewhere. 

3.  Fragile — environmental  conditions  or  accidental  mishaps  may  break  the 
equipment. 
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C.  CHAPTER  SUMMARY 

This  chapter  presented  the  examination  of  the  data  and  item  process  from  DLA 
DS  at  Camp  Pendleton,  California,  as  well  as  an  assessment  of  three  companies’ 
object/shape  recognition  technology.  Based  on  the  analysis  of  item  process,  the  data 
suggests  and  emphasizes  that  there  are  four  areas  of  concern  that  drive  inefficiency  and 
backlog.  Improperly  fdled  DD  1348  added  time  to  the  items  disposal  process  due  to 
researching  the  item’s  information;  RCN’s  items  are  limitless  and  backlogs  are  created 
due  to  accepting  greater  items  than  it  can  be  processed  qualify  for  factors  that  require  a 
solution  and  portrays  the  importance  of  this  study. 

Despite  previous  changes  to  streamline  the  steps,  the  current  process  can  be 
further  improved  to  reduce  or  eliminate  backlogs.  Generally,  automation  though  use  of 
technology  generates  favorable  conditions  that  can  be  considered  for  implementation. 
Currently,  the  private  and  government  sectors  have  some  form  of  object/shape 
recognition  technology  to  support  their  missions.  Of  the  three  companies’  technologies 
that  were  assessed  under  the  TRA  guidance,  Imaginestics  technology  is  the  most  viable 
and  mature  system,  ranking  up  to  a  TRL  5.  However,  the  standard  dictates  that  only  a 
TRL  7  or  above  is  mature,  and  therefore,  can  be  considered  for  implementation. 
Imaginestics  technology  has  the  potential  to  be  a  quickly  working  model  and  ready  for 
implementation  with  less  considerable  time  and  cost,  compared  to  the  other  two 
technologies  analyzed  here.  An  object/shape  recognition  tool  offers  benefits,  such  as 
reducing  property  identification  time,  when  a  well-designed  infrastructure  is  in  place. 
However,  consideration  of  upfront  cost  must  be  analyzed  to  justify  the  investment  and 
determine  whether  it  will  be  sustainable  in  the  long  run.  The  focus  of  Chapter  V  is  the 
CBA  that  presents  financial  aspect  of  investing  in  object/shape  recognition  technology. 
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V.  COST-BENEFIT  ANALYSIS 


The  objective  of  this  chapter  is  to  examine  whether  the  expected  benefits  of 
implementing  object/shape  recognition  technology  justify  the  cost.  The  analysis  generates 
a  measure  of  the  expected  economic  return  on  investment,  and  evaluates  the  feasibility 
that  object/shape  recognition  technology  can  be  installed  at  DLA  DS  at  Camp  Pendleton. 
The  CBA  is  consistent  with  the  U.S.  government  guidance:  Circular  A-94  Guidelines  and 
Discount  Rates  for  Benefit-Cost  Analysis  of  Federal  Programs  (OMB,  1992),  and  DOD 
Directive  5000.2  Interim — supportability  analysis  (DOD,  2013a).  The  benefits  are 
estimated  as  cost-savings  from  reduction  in  time  to  dispose  items  resulted  from 
implementing  object/shape  recognition  technology  to  the  item  disposal  process. 

This  CBA  analysis  evaluates  two  competing  alternatives:  the  status  quo,  which  is 
labor  intensive,  and  the  option  of  investing  in  object/shape  recognition  technology  for  the 
purpose  of  replacing  the  status  quo.  This  study  is  an  ex  ante  CBA-type  conducted  prior  to 
the  project  in  order  to  show  whether  it  is  worth  investing  in  object/shape  recognition 
technology  and  to  provide  infonnation  to  DLA  DS  decision-maker  prior  to  making  the 
investment.  Notwithstanding,  there  are  nine  basic  steps  in  conducting  a  CBA,  described 
as  follows: 

Step  1:  Specify  the  set  of  alternative  projects.  DLA  DS  currently  operates  the 
property  process  without  technology;  therefore,  this  alternate  is  considered  status  quo  and 
maintains  DLA  DS  property  identification  in  its  current  state.  The  second  alternate  will 
be  evaluates  and  analyzes  operations  with  technology. 

Step  2:  Decide  whose  benefits  and  costs  count  (standing).  The  stakeholders  are 
identified  as  follow:  DLA  DS  decision-makers  consider  the  cost  and  the  benefit  of  the 
alternatives.  DLA  DS  employees  are  directly  impacted  by  the  decisions  because  they 
conduct  the  property  identification  process.  The  U.S.  Congress  mandated  that  DLA  DS  to 
cut  operating  costs,  and  taxpayers  continue  to  fund  the  U.S.  government  through  taxes. 
This  thesis  focuses  on  providing  information  to  DAL  DS  decision-maker. 
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Step  3:  Catalog  the  impacts  and  select  measurement  indicators.  The  analysis 
result  includes  input  of  time  in  units  and  cost  of  labor.  Comparably,  the  focus  is  then 
shifted  to  associate  cost  of  labor  with  and  without  technology  in  order  to  measure  cause 
and  affect  outcomes.  The  results  should  provide  benefit  and  cost  impact. 

Step  4  :  Predict  the  impacts  quantitatively  over  the  life  of  the  project.  The  time 
horizon  is  limited  to  10  years  to  discount  the  cost  of  technology  depreciation  and 
replacement.  There  is  uncertainty  beyond  the  10-year  period,  and  to  capturing  that 
prediction  is  complex. 

Step  5:  Monetize  (attach  dollar  values  to)  all  impacts. 

Time  as  a  variable  is  monetized  in  order  to  provide  the  return  on  investment  and 
to  make  the  baseline  for  the  “willingness  to  pay.”  The  researcher  detennined  that 
investing  in  object/shape  recognition  technology  would  not  incur  revenues,  and  the 
investment  is  for  only  one  facility,  thereby  not  being  a  big  acquisition  program  to  raise 
revenues.  The  nature  to  justify  the  investment  is  costs  saved.  The  DLA  DS  does  not  make 
revenues,  given  that  it  is  part  of  the  DOD;  rather,  the  benefits  is  the  money  saved  by 
reducing  time  task  and  increasing  productivity.  Therefore,  the  employees’  work-time  is 
monetized  and  considered  only  when  the  particular  employee  uses  the  technology  to 
reduce  task  time  researching  items,  consequently  increasing  productivity. 

The  researcher  calculated  the  annual  wages  for  a  GS-11  in  dollars  per  hours  in 
order  to  monetize  the  LSN  research  task,  such  that  the  assumption  is  that  automating  the 
identification  process  with  technology  will  save  money  in  labor  cost.  The  Verifier  at 
Camp  Pendleton  is  a  GS-11  whose  annual  wages  is  between  $50,790  and  $66,027  and 
spends  half  the  day  researching  items’  information.  Therefore,  this  study  calculated  the 
following  information,  as  depicted  in  Table  8. 
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Table  8.  The  Annual  Cost  of  Researching  Items  Is  $27,282 


DLA  DS  Billet:  GS  1 1  Pay 

$50,790.00  -  $66,027.00  /  Per  Year 

For  this  study  $59,255  is  utilized  for  calculations 

Description 

Result 

Formula 

$  per  hour 

$28.49 

59255  ann  wgs  /(8hr  day  *  5  days)*52wks 

Mins  spent  on  1 7  items 

221 

1 3  min  *  1 7  items 

Hrs  per  day  spent  researching 

3.68 

221  min/60  in  hr 

Cost  per  day 

$104.93 

$28.49  per  hr  *  3.68  hrs  per  day 

Annual  cost  researching 

$27,281.99 

($105  cost  per  day*5days)  52  wks 

Table  8  illustrates  calculations  that  result  in  $27,282  a  year  to  research  items’ 
information.  The  calculation  accounts  for  a  GS-11  Step  6  who  dedicates  almost  four 
hours  a  day  with  items  that  need  furthers  steps  to  research  information  on  one  of  three 
databases  to  properly  dispose  the  items.  The  effect  of  the  change  in  cost  is  variable  due  to 
the  levels  of  GS  with  step  wage  an  employee  may  hold.  Furthermore,  the  scope  of  this 
thesis  limits  the  amount  of  variables  one  can  research  and  utilize  for  accounting  how  any 
other  income  from  other  variable  may  generates  more  market  comparable  revenues. 
However,  the  net  present  value  (NPV)  utilizes  the  final  cost  of  $27,282  as  a  fonn  of 
benefits. 

Step  6:  Discount  benefits  and  costs  to  obtain  present  values.  This  analysis  took 
into  account  a  2.5%  social  discount  for  labor  pay-raise  increases.  Additionally,  a  nominal 
discount  rate  is  used  to  discount  cost  and  benefit  over  the  10-year  horizon  time.  Thus,  in 
order  to  account  for  the  time  value  money  this  research  used  a  7%  discount  rate. 
According  to  the  Circular  A-94,  when  analyzing  proposed  investment  the  7%  rate  is  ideal 
for  calculating  the  return  on  investment  in  the  private  sector  (OMB,  1994). 

Step  7:  Compute  the  net  present  value  of  each  alternative.  The  NPV  equals  the 
present  value  of  benefits  minus  the  present  value  of  costs: 

NPV  =  PV  (B)  -  PV(C) 


Therefore,  corresponding  with  the  formula  the  cost  of  status  quo  is  in  the  form  of 
the  opportunity  cost,  such  that  by  leaving  the  item  disposal  process  as  is  DLA  DS  misses 


87 


the  opportunity  to  increase  productivity;  thus,  the  alternative  with  technology  reaps  the 
benefits  in  the  form  of  money  saved  by  reducing  time  task  and  increasing  productivity. 

Nevertheless,  there  were  challenges  of  gathering  actual  technology  cost  posed  by 
the  companies’  leadership,  due  to  proprietary  technology  rights,  privacy  policies,  and 
cautious  in  advertising  actual  cost  without  negotiating  a  contract.  But  discussions  to 
estimate  cost  in  a  theoretical  nature  were  provided  by  one  company  in  order  to  conduct 
the  thesis.  Therefore,  the  minimum  cost  to  buy  object/shape  recognition  is  about 
$150,000  for  one  facility.  This  includes  hardware,  software,  and  licensing.  The  intent  of 
the  study  is  to  offer  decision-makers  as  possible  related  information  to  what  the 
commercial  sector  has  to  offer  on  object/shape  recognition  technology;  so  under  the 
circumstances  this  is  the  best  estimate  provided  to  conduct  the  study.  Nonetheless,  the 
steps  and  procedures  are  consistent  with  the  guidance  such  that  it  can  be  used  to  further 
exploit  the  actual  cost  with  an  accurate  quote. 

In  order  to  calculate  the  NPV,  a  Monte  Carlo  model  was  utilized.  The  model 
accounts  for  risk  associated  with  future  uncertainty  of  the  2.5%  social  discount  for  labor 
pay-raise  increases,  which  may  be  approved  by  the  U.S.  government.  Therefore,  it  turns 
out  that  the  NPV  for  technology  provides  greater  benefit  than  status  quo. 

Step  8:  Perform  sensitivity  analysis.  A  sensitivity  analysis  supports  the  CBA  by 
changing  the  wages  as  a  form  of  revenues  assumptions  to  understand  risk  factors  that 
investing  in  object/shape  recognition  technology  inherits.  Table  9  illustrates  wages  a 
maximum  or  minimum  in  terms  of  revenues.  To  account  for  the  uncertainty  of  the 
predicted  impact,  this  study  conducted  a  yearly  variance  sensitivity  analysis  using  the  5% 
lower  bound  in  the  NPV’s  confidence  interval  that  resulted  with  $56,818  and  the  90% 
upper  bound  NPV’s  confidence  interval  resulting  in  $66,536.  The  minimum  and 
maximum  values  for  NPV  are  at  $53,732  and  $70,400,  respectively.  Thus,  NPV  remains 
positive  with  the  variations  in  wages  percentage  increase  or  decrease.  Appendix  C 
illustrates  the  sensitivity  analysis  that  supports  the  CBA  by  changing  the  wages  social 
discount  as  a  form  of  revenues  from  pay  raise  or  cut  between  the  time  value  money 
discount  rate  for  assumptions  in  order  to  understand  risk  factors  in  investing  in 

object/shape  recognition  technology  inherit. 
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Data 


Table  9.  Net  Present  Value  Results 


Initial  Investment 

$  150,000 

Cash  Flow  Year  1 

$  27,282 

Growth  rate  'g'  (1%  Std  Dev) 

2.50% 

Discount  rate  *r* 

7% 

Time  (yrs) 

10 

Fiscal  Year 

2014 

2015 

2016 

2017 

2018 

2019 

2020 

2021 

2022 

2023 

2024 

J  Year 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Growth 

3.79% 

2.28% 

2.47% 

4.03% 

2.90% 

1.61% 

2.70% 

4.56% 

2.69% 

3.78% 

Cash  Flows  (CF) 

-$  150,000 

$ 

27,282 

$ 

27,904 

$28,594 

$29,747 

$30,608 

$31,102 

$31,941 

$33,397 

$34,294 

$35,591 

Discount  CF  FACTOR 

1 

0.93 

0.87 

0.82 

0.76 

0.71 

0.67 

0.62 

0.58 

0.54 

0.51 

Discount  cash  Flow 

-$  150,000 

$ 

25,497 

$ 

24,372 

$23,341 

$  22,694 

$21,823 

$20,725 

$  19,891 

$  19,437 

$18,654 

$18,093 

Cumulative  Discount  CF 

-S  124,503 

-$  100,131 

-$  76,790 

-$  54,096 

-$  32,273 

-$  11,548 

$  8,343 

$  27,780 

$46,434 

$64,527 

Net  Present  Value  Results: 


NPV 

$ 

64,527 

Internal  Rate  of  Return  (IRR) 

15.13% 

Profitability  Index  (PI) 

$ 

1.43 

Pay  Back  (years) 

6.7 

NPV  -  Monte  Carlo  Simulation  Results 

Number  of  Samples; 

1000 

NPV  Mean 

$ 

62,265 

NPV  -  Standard  Deviation 

$ 

3,397 

Minimum 

$ 

53,732 

Maximum 

$ 

70,400 

5%  Percentile 

$ 

56,818 

90%  Percentile 

$ 

66,536 

Step  9:  Make  a  recommendation. 

The  recommendation  to  implement  object/shape  recognition  is  viable;  however  it 
will  solve  the  less  than  1%  of  unlabeled  items  and  not  solve  the  7.5  days’  worth  of 
backlog  items  for  the  DLA  DS  facility  located  in  Camp  Pendleton.  The  estimated  cost  to 
implement  one  object/shape  recognition  system  is  about  $150,000.  The  total  quantified 
net  economic  benefit  occurring  from  the  technology’s  contributions  is  estimated  to  be 
about  $65,000  when  applying  a  7%  discount  rate,  and  a  2.5%  social  discount  for  labor 
pay-raise  increases  the  corresponding  internal  rate  of  return  results  at  15.13%  — making 
the  investment  desirable  to  pursue,  and  profitability  index  (PI)  1.43,  indicating  that  for 
every  dollar  invested,  $1.43  in  benefits  is  accrued  with  a  payback  period  6.7  years  over  a 
10-year  time  span.  Applying  a  7%  discount  rate  yields  a  NPV  of  $35,591.  The  payback 
period  is  illustrated  in  Figure  28. 
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A.  SUMMARY 

The  benefit  from  implementing  object/shape  recognition  technology  outweighs 
the  cost  to  purchase  it.  The  new  technology  will  save  money  in  labor  costs.  The  10-year 
period  of  revenues  provided  by  the  technology  recoups  the  $150,000  initial  investment  in 
about  seven  years  such  that  the  internal  rate  of  return  of  1 5%  is  greater  than  the  cost  of 
capital  estimated  at  1%.  The  profitability  index  is  $0.43  to  each  one  dollar  invested, 
suggesting  that  DLA  DS  should  execute  the  investment  and  pursue  it  for  further 
maturation.  However,  the  scope  of  this  thesis  to  research  more  areas  of  cost  and  benefits 
was  limited  to  considering  the  new  technology  obtain  only  as  an  alternative  to  the  status 
quo.  In  addition,  the  analysis  used  estimates  of  some  of  the  elements  of  costs,  as 
inaccurate  costs  of  new  technologies  are  not  available.  However,  the  CBA  developed  in 
this  thesis  can  be  easily  adjusted  to  incorporate  more  accurate  data  once  it  becomes 
available  and  therefore  allow  a  certified  contracting  officer  to  pursue  further  the  validity 
of  an  object/shape  recognition  technology  option. 
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VI.  CONCLUSION 


A.  INTRODUCTION 

This  chapter  summarizes  the  project — a  quest  for  answers  to  the  problem  that 
DLA  DS  at  Camp  Pendleton  faces.  Provide  recommendations  to  the  four  areas  of 
concerns.  And,  brief  areas  of  uncertainty  that  would  require  further  study. 

The  Lean  Six  Sigma  study  DLA  DS  conducted  in  2008  resulted  streamlining  the 
item-disposal  process  by  saving  the  time  equivalent  of  40%  and  concluded  to  conduct 
further  study  to  exploit  the  operation  process  with  technology.  This  thesis  aimed  to 
address  the  suggested  new  technology  implementation  option.  Specifically,  the  thesis 
addressed  the  following  research  questions.  Answering  these  questions  was  the  key  take 
away  and  culmination  of  the  research. 

1 .  Is  there  an  object/shape  recognition  technology  to  support  the  feasibility 
of  implementing  it  at  DLA  Disposition  Services? 

2.  Secondary  key  questions  to  answer  include  the  following:  (1)  What  are 
the  technology  needs  for  DLA  Disposition  Services?  (2)  Is  the 
technology  mature  enough  for  implementation?  (3)  Where  will  the 
technology  and  equipment  be  obtained?  (4)  How  much  will  the 
equipment  and  technology  cost  and  will  it  generate  any  tangible  savings? 
(5)  What  is  the  payback  period? 

To  answer  the  research  questions  this  study  utilized  a  retrospective  mixed  mode 
analysis  of  DLA  DS’s  items  disposal  processing  using  data  collected  from  Camp 
Pendleton,  California,  a  technology  readiness  assessment  (TRA)  on  object/shape 
recognition  technology  maturity  level,  and  CBA  to  examine  the  feasibility  to  implement 
object/shape  recognition  technology.  The  technology  readiness  assessment  and  the  CBA 
are  consistent  with  the  Circular  A-94  (OMB,  1994),  DOD  Directive  5000.2  Interim 
(DOD,  2013a)  and  DOD  TRA  (Assistant  Secretary  of  Defense  for  Research  and 
Engineering  [ASD(R&D)],  2011)  guidance  to  calculate  the  object/shape  recognition 
technology’s  benefits  of  reducing  item  research  time.  The  study  is  retrospective  in  that  it 
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included  and  assessed  only  a  four-month  period  of  data  and,  given  that  DLA  DS  is  a  non¬ 
profit  organization,  the  researcher  utilized  a  “market  comparable  approach”  to  calculate 
the  benefits,  costs,  and  return  on  investment.  Although  DLA  DS  does  not  have  an  overall 
comparable  commercial  market  force  to  compare  revenues,  this  approach  allowed  the 
researcher  to  use  the  market  comparable  labor  as  a  form  of  estimating  revenues. 

B.  PROJECT  SUMMARY 

This  thesis’  objective  was  to  understand  the  items  disposal  process  and  assess 
object/shape  recognition  technology  to  address  the  issue  unlabeled  items  poses  to  DLA 
DS.  However,  the  findings  generated  by  the  analysis  of  DLA  DS’  items  disposal 
processing  show  three  other  areas  of  concern  that  contribute  to  days’  worth  of  backlog 
items.  The  analysis  suggests  that  the  problem  is  not  exactly  the  unlabeled  items.  Indeed, 
employees  spend  excess  time  conducting  research  on  items  that  are  not  labeled  or 
documented  on  the  DD  1348  fonn.  However,  unlabeled  items  are  less  than  1%  of 
accepted  items  and  it  does  not  fully  contribute  to  days’  worth  of  backlog.  Three  other 
area  of  concern  contributed  to  days’  worth  of  backlog.  First,  improperly  filled  DD  1348 
provides  for  additional  time  spent  conducting  research  in  order  to  fill  the  DD  1348  form 
for  further  processing.  Second,  the  receipt  control  numbers  (RCNs)  are  not  limited  to  the 
amount  of  items  that  can  arrive  in  a  pallet  or  tri-box.  A  RCN  can  have  1  or  50  items.  This 
creates  the  potential  to  accept  an  average  of  425  items  in  one  day  and  DLA  DS  at  Camp 
Pendleton  capacity  to  process  items  is  limited  to  300  items  with  six-employee  crew. 
Therefore,  third,  DLA  DS  at  Camp  Pendleton  accepts  more  items  than  what  it  can 
process,  which  results  in  having  an  average  of  over  2,000  items  in  the  backlog  stowage 
area.  The  facility  processes  an  average  300  items  a  day  and  had  an  average  47  RCN 
equaling  to  an  average  of  2,259  items  or  7.5  days’  worth  of  backlog  items  stored  for  later 
processing. 

This  thesis  examined  the  alternative  of  addressing  the  unlabeled  items  issues  with 
the  use  of  new  object/shape  recognition  technology.  The  DLA  DS  technological 
requirement  includes  item  recognition  and  disposal  infonnation  feedback,  and  the  ability 
to  take  pictures  to  use  on  DLA  email  system  to  communicate  and  upload  the  picture  to 
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liquidations  website.  This  thesis  assessed  the  readiness  level  of  existing  object/shape 
recognition  technology  by  analyzing  three  available  such  technologies: 

First,  technologies  that  are  ready  with  DLA  DS’s  specifications  and  operate  in 
DLA  DS  similar  environment  have  not  been  created,  therefore  none  meet  the  required 
threshold  of  TRL  7  or  above. 

Cognex  designed  machines  specific  to  customers’  requirement  and  the  company 
does  not  have  a  machine  ready  for  DLA  DS  to  operate  straight  out  of  the  box,  therefore 
Cognex  TRL  is  2.  To  implement  Cognex’s  technology  to  DLA  DS  it  will  require  further 
maturation  of  technology  costing  excess  time  and  funding  making  Cognex  technology 
less  desirable  to  be  adopted  by  DLA  DS. 

Imaginestics  technology  is  an  application  for  mobile  devices  and  requires  Internet 
to  access  the  database  for  feedback.  Lockheed  Martin  is  utilizing  Imaginestics’ 
technology.  The  DOD  funded  Imaginestics’  technology.  Furthermore,  Imaginestics  has 
been  approved  for  information  assurance  (IA)  for  another  technology  under  the  Navy  and 
Office  of  the  Secretary  of  Defense  (OSD),  therefore  the  company  understand  the  DOD 
acquisition  and  IA  approval  process.  However,  Imaginestics  object/shape  recognition 
technology  it  ranks  at  TRL  5,  but  Imaginestics  technology  rates  higher  TRL  amongst  the 
three  companies  making  it  more  desirable  to  pursue  further  maturation  of  Imaginestics 
technology.  Investing  in  Imaginestics  technology  will  cost  less  time  and  funding 
compared  to  the  other  two  companies. 

RMD  ’s  automated  digital  recognition  technology  (AD ART)  was  artificial 
intelligence  tool  designed  for  the  Marine  Corps  to  capture  and  recognize  mechanics’ 
repair  process  for  identification  and  feedback.  However,  the  funding  to  further  mature  the 
technology  was  stalled  and  the  machine  is  still  in  a  prototype  raking  in  TRL  2.  To 
implement  RMD’s  technology  to  DLA  DS  it  will  require  further  maturation  of 
technology  costing  excess  time  and  funding  making  RMD  technology  less  desirable  to  be 
adopted  by  DLA  DS. 

The  benefit  from  implementing  object/shape  recognition  technology  outweighs 
the  cost  to  purchase  it.  The  new  technology  will  save  money  in  labor  costs  for  the 
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unlabeled  items  that  are  less  than  1%  of  accepted  items.  The  10-year  period  of  revenues 
provided  by  the  technology  recoups  the  $150,000  initial  investment  in  about  seven  years 
such  that  the  internal  rate  of  return  of  15%,  which  is  greater  than  the  cost  of  capital 
estimated  at  7%.  The  profitability  index  is  $0.43  to  each  one  dollar  invested,  suggesting 
that  DLA  DS  should  execute  the  investment  and  pursue  it  for  further  maturation. 
However,  implementing  object/shape  recognition  technology  does  not  fully  contribute  the 
reduction  of  days’  worth  of  backlog.  Therefore,  the  results  from  the  queuing  theory 
suggests  that  with  nine  employees  the  DLA  DS  at  Camp  Pendleton  will  keep  up  with 
demand  by  processing  all  received  items  and  will  progressively  reduce  the  7.5  days’ 
worth  of  backlog. 

One  of  the  main  conclusions  resulted  from  this  thesis  effort  is  that  researching 
items  is  not  the  only  area  of  concern  with  regard  to  process  inefficiency.  Rather,  this 
thesis  identified  three  other  areas  of  concern  suggested  by  the  data  analysis  and 
observation.  Therefore,  further  recommendations  were  provided  to  take  care  of  the  other 
three  problems. 

C.  RECOMMENDATIONS 

Given  the  main  findings  identified  by  this  thesis  analysis,  the  recommendation  to 
implement  object/shape  recognition  is  viable  as  far  as  generating  a  positive  return  to 
investment;  however  it  only  address  less  than  1%  of  items  and  not  the  7.5  days’  worth  of 
backlog  items  for  the  DLA  DS  facility  located  in  Camp  Pendleton.  Investing  in 
Imaginestics’  technology  for  further  maturation  will  ease  the  disposal  process  for  the  1% 
of  unlabeled  items. 

Although  this  study  recommends  object/shape  recognition  to  alleviate  the  1%  of 
accepted  items,  DLA  DS  must  tackle  the  other  three  areas  of  concern  with: 

1.  Making  mandatory  to  fill  the  DD  1348  on  DLA  DS  website,  and 

2.  Limiting  the  amount  of  items  on  pallets  to  match  the  facility’s  processing 
capacity,  and 

3.  Increasing  the  Camp  Pendleton  facility  to  nine  employees,  or 
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4.  Adding  a  night  shift  such  that  the  shift  only  process  items  without 
accepting  or  conducting  mundane  tasks,  or 

5.  Once  a  month  stop  accepting  items  for  disposal  process  for  a  full  week  in 
order  to  focus  the  effort  to  dispose  the  backlog  items,  or 

6.  Fill  the  gap  with  technology  by  implement  full  automation  with  optical 
sorting  and  data  mining  that  included  sensors,  laser,  object/shape 
recognition  technology  on  conveyor  belt  system  and  data  analytics  to 
improve  materiel’s  opportunity  to  be  sold  as  commodity  or  liquidations 
and  improve  operational  efficiency;  which  would  be  consistent  with  DLA 
Director’s  “Big  Idea”  and  strategic  technological  goals  to  accomplish. 

The  technological  innovation  disrupts  status  quo  by  ensuring  a  plan  of  increased 
capacity  for  the  long  run  and  perhaps  reduces  some  of  the  over  $60  million  of  labor  cost 
that  account  for  about  35%  of  DLA’s  budget. 

D.  RECOMMENDATION  FOR  FUTURE  RESEARCH 

Areas  of  uncertainty  remain  for  future  study.  This  study  was  not  able  to  capture 
time  spent  in  every  step  of  the  item  disposal  process,  or  observe  whether  the  backlog  is  a 
cyclical  trend  due  to  retrograding  units  from  deployments,  cyclical  due  to  time  of  the 
year,  and  value  added  from  items  that  are  scrapped  rather  than  reaping  on  the  opportunity 
to  be  sold  and  increase  revenues.  For  example  if  the  technology  increases  the  opportunity 
to  send  more  items  for  liquidations  the  opportunity  would  be  in  the  benefit  of  increasing 
revenues  to  DLA.  Additionally,  the  scope  of  the  thesis  narrowed  the  study  the  Camp 
Pendleton  DLA  DS  facility,  such  that  the  results  may  not  be  indicative  of  all  DLA  DS. 
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APPENDIX  A.  DISCLAIMER 


The  following  disclaimer  was  adopted  from  the  Department  of  Energy  (DOE, 
2012)  because  it  is  relevant  to  this  study: 

This  report  was  prepared  as  an  account  of  work  sponsored  by  an  agency  of 
the  United  States  Government.  Neither  the  United  States  Government  nor 
any  agency  thereof,  nor  any  of  their  employees,  makes  any  warranty, 
express  or  implied,  or  assumes  any  legal  liability  or  responsibility  for  the 
accuracy,  completeness,  or  usefulness  of  any  information,  apparatus, 
product,  or  process  disclosed,  or  represents  that  its  use  would  not  infringe 
privately  owned  rights.  Reference  therein  to  any  specific  commercial 
product,  process,  or  service  by  trade  name,  trademark,  manufacturer,  or 
otherwise  does  not  necessarily  constitute  or  imply  its  endorsement, 
recommendation,  or  favoring  by  the  United  States  Government  or  any 
agency  thereof.  The  view  and  opinions  of  authors  expressed  therein  do  not 
necessarily  state  or  reflect  those  of  the  United  States  Government  or  any 
agency  thereof  (p.  iii). 
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APPENDIX  B.  NORMALIZED  DATA 


Jul 

through.... 

7/31/14 

sorted  by 
Region,  Site 
&  Date 

Point  in 

Time 

RCN  Open 

Point  in 

Time 

RCN  AGE 

RCN  to 

Induct 
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Point  in 

Time 

Backlog 

Ending 

(Lines) 

Receipts 
(Lines  + 
Batches) 

Point  in  Time 

ADD  New  DAILY 

Data  to  bottom 
(Paste  Values), 
then  Sort  by 

Region,  Site, 

Date 

Backlog  Ratio 

Inventory 

Estimete 

processing 

Results 

Date 

Region 

Site  ID 

Site 

Count 

Oldest  RCN 

Days  Avg 

Backlog 

Count 

Receipts 

Ratio 

7/1/14  West 

RCCP 

DLA  DS  PEND 

30 

12 

6.6 

1,082 

466 

232% 

7/2/14  West 

RCCP 

DLA  DS  PENDLI 

30 

12 

8.6 

1,104 

256 

431% 

7/3/14 

West 

RCCP 

DLA  DS  PENDLI 

30 

12 

8.6 

1,703 

375 

454% 

7/4/14  West 

RCCP 

DLA  DS  PENDLI 

30 

1 2 

<7 

7/5/14  West 

RCCP 

DLA  DS  PENDLI 

50 

12 

0 

7/6/14  West 

RCCP 

DLA  DS  PENDLI 

Z8 

12 

1.703 

0 

7/7/14  West 

RCCP 

DLA  DS  PEN! 

31 

13 

9.9 

2,320 

231 

1004% 

7/8/14  West 

RCCP 

DLA  DS  PENI 

31 

14 

10.0 

2,417 

357 

677% 

7/9/14  West 

RCCP 

DLA  DS  PENI 

34 

13 

7.9 

2,507 

311 

806% 

7/10/14  West 

RCCP 

DLA  DS  PENI 

37 

14 

3.6 

2,530 

298 

849% 

7/11/14  West 

RCCP 

DLA  DS  PENI 

37 

14 

8.5 

2,538 

273 

930% 

7/12/14  West 

RCCP 

DLA  DS  PENI 

37 

14 

- 

- 

fo 

7/13/14  West 

RCCP 

DLA  DS  PENI 

40 

17 

- 

2,538 

0 

7/14/14  West 

RCCP 

DLA  DS  PENI 

44 

15 

14.3 

2,353 

351 

670% 

7/15/14  West 

RCCP 

DLA  DS  PENI 

47 

16 

12.5 

2,108 

333 

633% 

7/16/14  West 

RCCP 

DLA  DS  PENI 

51 

17 

13.5 

2,055 

303 

678% 

7/17/14  West 

RCCP 

DLA  DS  PENI 

51 

18 

12.5 

2,099 

301 

697% 

7/18/14  West 

RCCP 

DLA  DS  PENI 

51 

18 

10.5 

2,544 

232 

1097% 

7/19/14  West 

RCCP 

DLA  DS  PENI 

51 

18 

1MJI 

401 

0% 

7/20/14  West 

RCCP 

DLA  DS  PENI 

45 

18 

- 

2,544 

0 

7/21/14  West 

RCCP 

DLA  DS  PENI 

47 

19 

15.4 

2,446 

157 

1558% 

7/22/14  West 

RCCP 

DLA  DS  PENI 

54 

16 

13.4 

2,431 

213 

1141% 

7/23/14  West 

RCCP 

DLA  DS  PENI 

59 

17 

6.4 

2,410 

107 

2252% 

7/24/14  West 

RCCP 

DLA  DS  PENI 

51 

18 

4.9 

2,273 

230 

988% 

7/25/14  West 

RCCP 

DLA  DS  PENI 

51 

18 

6.4 

2,456 

401 

612% 

7/26/14  West 

RCCP 

DLA  DS  PENI 

51 

18 

- 

- 

0 

7/27/14  West 

RCCP 

DLA  DS  PENI 

52 

21 

- 

2,456 

ro 

7/28/14  West 

RCCP 

DLA  DS  PENI 

47 

22 

10.2 

2,313 

508 

455% 

7/29/14  West 

RCCP 

DLA  DS  PENI 

49 

23 

ll7 

2,383 

432 

552% 

7/30/14  West 

RCCP 

DLA  DS  PENI 

53 

24 

6.9 

2,601 

520 

500% 

7/31/14  West 

RCCP 

DLA  DS  PENI 

62 

25 

13.7 

3,072 

140 

2194% 

Figure  29.  Normalized  Data  Treated  with  Excel’s  Default  Setting  “Show  empty  data 

as  zero” 
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APPENDIX  C.  NPV  SENSITIVITY  ANALYSIS  TABLE 


Sensitivity  Analysis  -  NPV 

Discount  Rate  V 

5% 

6% 

7% 

8% 

9% 

10% 

11% 

12% 

13% 

14% 

15% 

2% 

$ 

78,844 

$ 

67,792 

$ 

57,520 

$ 

47,963 

$ 

39,058 

$  30,752 

IXS1 

11 

$ 

8,954 

$ 

2,594 

-$  3,373 

3% 

$ 

88,652 

$ 

76,954 

$ 

66,088 

$ 

55,983 

$ 

46,574 

$  37,802 

$  29,616 

$ 

21,966 

$ 

14,810 

$ 

8,108 

$  1,825 

4% 

$ 

98,971 

$ 

86,588 

$ 

75,094 

$ 

64,409 

$ 

54,467 

$  45,203 

$  36,562 

a 

a 

$ 

13,887 

$  7,270 

5% 

$ 

109,828 

$ 

96,721 

$ 

84,560 

$ 

73,264 

$ 

62,757 

$  52,973 

$  43,852 

$ 

35,338 

$ 

27,383 

$ 

19,941 

$  12,972 

fZ rniA/f n  'o' 

6% 

$ 

121,252 

$ 

107,377 

$ 

94,512 

$ 

82,567 

$ 

71,463 

$  61,129 

$  51,500 

$ 

42,518 

$ 

34,129 

$ 

26,286 

$  18,945 

VjlUWUl  5 

7% 

$ 

133,269 

$ 

118,583 

$ 

104,972 

$ 

92,341 

$ 

80,606 

$  69,691 

$  59,526 

$ 

50,048 

$ 

41,201 

$ 

32,935 

$  25,202 

8% 

$ 

145,912 

$ 

130,367 

$ 

115,967 

$ 

102,611 

s 

90,209 

$  78,679 

$  67,947 

$ 

57,946 

$ 

48,616 

$ 

39,902 

$  31,755 

9% 

$ 

159,211 

$ 

142,757 

$ 

127,523 

$ 

113,401 

$ 

100,293 

$  88,114 

$  76,783 

$ 

66,230 

$ 

56,390 

$ 

47,204 

$  38,620 

10% 

$ 

173,200 

$ 

155,785 

$ 

139,668 

$ 

124,736 

$ 

110,884 

$  98,018 

$  86,055 

$ 

74,919 

$ 

64,540 

$ 

54,856 

$  45,812 

11% 

$ 

187,914 

$ 

169,482 

$ 

152,433 

$ 

136,644 

$ 

122,004 

$  108,414 

$  95,784 

$ 

84,032 

$ 

73,084 

$ 

62,876 

$  53,345 

12% 

$ 

203,388 

$ 

183,881 

$ 

165,847 

$ 

149,153 

$ 

133,682 

$  119,326 

$  105,991 

$ 

93,589 

$ 

82,042 

$ 

71,280 

$  61,237 

Figure  30.  A  Sensitivity  Analysis  of  Wages  between  Social  Discount  Rate  (Growth  “g”)  and  Discount  Rate  “r” 
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Figure  30  illustrates  the  sensitivity  analysis  that  supports  the  CBA  by  changing 
the  wages  social  discount  as  a  form  of  revenues  from  pay  raise  or  cut  between  the  time 
value  money  discount  rate  for  assumptions  in  order  to  understand  risk  factors  in  investing 
that  object/shape  recognition  technology  inherit. 
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